1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
|
Standalone Setup
----------------
NOTE: The standalone setup is most suitable for development and is not
recommended for production; for production use the Docker-based setup
covered in the main README.
In order to run this application standalone, you will need:
* Python 3.6+
* Django 3.2.x (not compatible with 3.0 or older)
* RabbitMQ 3.8.x
* For production usage, a web server set up to host Django applications
(not needed for local-only testing/development)
* A database system supported by Django (SQLite, MariaDB/MySQL, etc.).
* Third-party Django/python modules as detailed by requirements.txt.
If you're using pip, you can install these by running:
pip install -r requirements.txt
(See below for a virtualenv-based setup which is easy to work with.)
* On the machine that will run the backend update script (which does not
have to be the same machine as the web server, however it does still
have to have Django installed, have the same or similar configuration
in settings.py and have access to the database used by the web
application):
* Python 3.5+ to match with the version of BitBake
for the OpenEmbedded branch being parsed
* Python dependencies as per requirements.txt (we still need Django
etc. here since we interact with the database through Django's ORM.)
Example virtualenv-based setup for the above:
Python's virtualenv provides an easy way to isolate the python dependencies
of applications such as the layer index. Here's an example of setting up a
virtualenv for the layer index that's particularly useful for development.
(This assumes a Debian-based distribution, adjust accordingly for other
distros).
1. Install required host distro packages (some of these are required by
pip to build the dependencies; it's also assumed you want MariaDB as
the database backend):
sudo apt-get update
sudo apt-get install python3-venv libmariadb-client-lgpl-dev build-essential python3-dev libjpeg-dev libz-dev libfreetype6-dev libssl-dev mariadb-server rabbitmq-server
2. Work around path issues (you may not need this):
sudo ln -s /usr/bin/mariadb_config /usr/bin/mysql_config
3. Create a Python 3 virtualenv (path can be anywhere you like):
python3 -m venv /path/to/desired/venv
4. Activate the virtualenv:
. /path/to/desired/venv/bin/activate
5. Install requirements:
pip install -r requirements.txt
Setting up MariaDB/MySQL:
Assuming you wish to use MariaDB/MySQL for storing the database - the
following is completely standard, but to save you having to look it up
elsewhere:
1. Start the server:
sudo /etc/init.d/mysql start
2. Set the root password and secure the database server:
sudo mysql_secure_installation
3. Create a database (substituting "mydatabase" with desired database
name):
mysql -u root -p
CREATE DATABASE mydatabase CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
Configuration and running:
1. Edit settings.py to specify a database, EMAIL_HOST, SECRET_KEY and
other settings specific to your installation. Ensure you set
LAYER_FETCH_DIR to an absolute path to a location with sufficient
space for fetching layer repositories. Modify RABBIT_BROKER
and RABBIT_BACKEND to reflect the settings used by your RabbitMQ
server.
2. Run the following commands within the layerindex-web directory to
initialise the database:
./manage.py migrate
./manage.py createsuperuser
3. You can test the web application locally by setting DEBUG = True in
settings.py and running the following:
./manage.py runserver
Then visit http://127.0.0.1:8000/layerindex/ with your browser. As
with all Django applications there is an admin interface available
at http://127.0.0.1:8000/admin/ also. The initial login and password
will be those you entered in the step above when creating an admin
account.
NOTE: This local server should only be used for testing - for
production you need to use a proper web server and have DEBUG set
to False (or use the Docker-based setup).
3.1. In order to process asynchronous tasks like sending email,
you will need to run a Celery worker:
celery -A layerindex.tasks worker --loglevel=info
4. To import layer data from the public instance at layers.openembedded.org
you can run the following (defaults to the master branch only):
layerindex/tools/import_layers.py -d https://layers.openembedded.org
Alternatively you can add layers individually using the separate
layerindex/tools/import_layer.py script or through the admin interface.
SMTP test server
----------------
If you want to test email functionality (e.g. for password reset, layer
submission or other notifications) you can do so without dealing with actual
emails being sent by running python's built-in SMTP test server (in a
separate shell session):
python3 -m smtpd -n -c DebuggingServer localhost:1025
This will stay running and print out any email that is sent to it. All you
need to do then is edit settings.py and set it to point to the server:
EMAIL_HOST = 'localhost'
EMAIL_PORT = 1025
Updating OpenEmbedded data
--------------------------
You will likely want to update the OpenEmbedded layer information on a regular
basis. To do that:
Incremental update:
layerindex/update.py
Reload all data (should only be needed if the database structure has
changed as part of an upgrade of the application code):
layerindex/update.py -r
|
