aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)Author
2019-10-02Add a space between text and glyphiconspaule/dissector-backports3Paul Eggleton
Where we use glyphicons to mark items in a list, ensure there is a space between the item and the icon to make things look a bit neater. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-10-02requirements.txt: updatePaul Eggleton
* Bump a few versions where we can * Drop anyjson - this used to be a dependency of kombu but not anymore, and nothing else needs it. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-10-02README: add basic documentation on db backup and restorePaul Eggleton
Make it a bit easier for the user to back up and restore the database. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-10-02update_classic_status: ensure fields are correctly orderedPaul Eggleton
On some systems it seems the order of fields in a values() query aren't guaranteed, so in order to avoid the contents dancing around too much, explicitly write out the fields in the order we expect them to be in. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-10-02README.devel: touch up a few thingsPaul Eggleton
* Drop some explicit outdated version numbers * You do actually have to create the database with MariaDB etc. so add instructions as to how for convenience * Add pointer to virtualenv in python requirements item * Add libssl-dev package to install command * Use python3-venv instead of virtualenv package in install command Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-10-02import_layers: tweak debug logging to be a bit more usefulPaul Eggleton
* Drop most "Skipping..." messages * Use layerbranch name instead of ID Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-10-02import_layers: use vcs_last_rev to check if record is updatedPaul Eggleton
Unfortunately the "updated" date gets changed every time the repo is fetched (i.e. vcs_last_fetch changes), but that doesn't mean the layerbranch or anything under it has actually changed. Use vcs_last_rev instead although unfortunately that won't catch manual edits to the layerbranch alone. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-10-02dockersetup: ensure letsencrypt volume change gets undonePaul Eggleton
If you enable the --letsencrypt option when you run dockersetup.py, the script will modify the volume mount for the certificates to point to /etc/letsencrypt instead of /opt/cert. If you then run dockersetup.py again (with -r/--reinstall) without --letsencrypt, we want the path to be set back to /opt/cert, so ensure that it does. Additionally, the code wasn't actually setting the path for the layerscertbot service since editing that section was done separately. (Admittedly, the letsencrypt functionality has not been well-tested.) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-10-02update_classic_status: add cover status import/exportPaul Eggleton
Provide a way to export and import other distro recipe coverage in update_classic_status.py. (Based on the Clear Linux Dissector commit ff38e9582093453e13b90e06af125374ca4b0a16). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-10-02update_classic_status: switch to argparsePaul Eggleton
I'm about to add a few command line options to this script, so switch over to argparse before that which is more modern and a bit more flexible. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-10-02update_classic_status: tweak single-object queriesPaul Eggleton
.first() returns None if the query is empty, so we can simplify this. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-10-02dockersetup: minor command line comment fixPaul Eggleton
We do allow compressed SQL files. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-10-02dockersetup: allow updating if proxy config set in environmentPaul Eggleton
Don't confuse proxy options being defaulted through from the environment and the user explicitly specifying them. Also look at no_proxy option. Fixes https://github.com/intel/clear-linux-dissector-web/issues/13 Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-10-02dockersetup: handle ArgumentTypeError properlyPaul Eggleton
Print ArgumentTypeError (which we are raising) instead of showing a traceback. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-10-02dockersetup: add uninstall optionPaul Eggleton
Add the ability to uninstall the application for the user's convenience. (Note that this does not undo the changes to the configuration, it only removes the Docker containers and volumes.) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-10-02dockersetup: proxy support fixesPaul Eggleton
* We need a SOCKS proxy to support fetching from git:// or ssh:// URLs within the container, so add an option to specify it * It's possible for the http and https proxy settings to be the same, so set one from the other if only one of them is set. * If we want to be able to fetch from internal servers inside the proxy then we also need a "no-proxy" list, so add support for that. * It's not unlikely that machines within networks requiring use of a proxy for external network access will have all of the proxy settings set in the environment, so we can try to pick up the defaults from there. * Ensure that we can switch from proxy to no proxy (when reinstalling) which means we always need to edit the config files and ensure the proxy options get commented out if we don't want them set. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-10-02docker: fix comment at top of DockerfilePaul Eggleton
docker/README is no more - the info is all in the main README now. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-10-02dockersetup.py: add connectivity checkPaul Eggleton
It's easy to get the proxy settings wrong and not realise until you've got quite a long way into the process of setting things up. Thus, add a check where we actually try to fetch various things within the container environment and fail reasonably early if things aren't working. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-09-18API: fix recipes view to really point to recipes ViewSetPaul Eggleton
I forgot the lesson of f5922091b4c8b888849df158548453b73f92a377 which was that djangorestframework has a very silly default of registering the model name as the key for finding the ViewSet, with the result that the recently created RecipesExtendedViewSet was also being used for the recipes URL which we expect to point to RecipeViewSet; this broke the bitbake-layers layerindex-* commands. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-09-17import_layers: fix updating build dependenciesPaul Eggleton
Apparently I forgot that filter() returns a QuerySet and not an actual object. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-09-16import_layers: fix output when importing layers from scratchpaule/restapi-fixesPaul Eggleton
Fixes "Importing None" message since we had not yet initialised the layerbranch variable. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-09-16import_layers: use recipesExtended viewsetPaul Eggleton
The new viewset has a different URL and uses pagination, so we need to take that into account. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-09-16API: fix recipes API performance regressionPaul Eggleton
Adding these extra child items to the standard "recipes" viewset (which we did recently in 684a06a383fd2a8da80490dff5f0dbc47750934e) means that some current usage is impractical due to the size of the returned list of items. Instead, create a recipesExtended viewset, move the new child items to that and add pagination to avoid result size issues. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-08-06README: add reference to using import_layers to updatepaule/importlayersPaul Eggleton
This is now be practical to use for updating as well as the initial import. Additionally the user doesn't need to be using the debug option by default so drop it from the example command. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-08-06import_layers: show basic progressPaul Eggleton
Show an info message with a counter when importing layers. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-08-06import_layers: add reload optionPaul Eggleton
Make it possible to reload the data even if the updated field says it's up-to-date. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-08-06import_layers: delete non-matching LayerBranch objectsPaul Eggleton
If a LayerBranch is on a branch that is in the remote layer index (and that branch is in any branch list specified with -b/--branch) and the layer for the LayerBranch is not found in the remote layerindex then it should be deleted, otherwise we could end up with stale data. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-08-06import_layers: allow filtering layersPaul Eggleton
Add an option to import only certain layers - mostly for debugging (can make testing a lot quicker). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-08-06import_layers: allow filtering branchesPaul Eggleton
We only work with branches that exist in both the local database and the remote index, but allow the user to restrict the list further if desired. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-08-06import_layers: switch to argparsePaul Eggleton
No functionality changes, just a code upgrade. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-08-06Add simple script for testing web APIPaul Eggleton
This is not a comprehensive test, but does provide a way to easily compare that what was originally fetched for a recipe from the API is the same once it's been imported into a new instance using import_layers. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-08-06admin: add layerbranch to BBAppend string representationPaul Eggleton
This makes it a little bit easier to see which BBAppend objects belong to which layer. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-08-06import_layers: import other missing itemsPaul Eggleton
Import Machine, Distro, BBClass, BBAppend, IncFile objects. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-08-06import_layers: import recipesPaul Eggleton
Import recipe data from the REST API, including sub-recipe objects (sources, patches, PACKAGE_CONFIGs, dependencies etc.) now that they are available from the API. Also, with the data being more or less complete, it's also reasonable to also set the vcs_* fields so that we can tell how up-to-date it is. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-08-05import_layers: enable incremental LayerItem/LayerBranch importPaul Eggleton
Don't assume we need to create these objects - check the updated field value against what the API reports and update the existing object if the latter is newer. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-08-05import_layers: avoid importing duplicate objectsPaul Eggleton
For LayerDependency, LayerMaintainer and LayerNote objects, properly sync the objects in the database with the objects fetched from the API. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-08-05API: add BBAppend and IncFile objects to REST APIPaul Eggleton
These are a little less useful than the other items but if we want to be able to create a clone of a layer index then we need them. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-08-05API: add sources, patches, dependencies to recipesPaul Eggleton
We're tracking these items for each recipe, so they should be accessible via the API as well. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-31dockersetup: fix regression with https enabledpaule/dissector-backportsPaul Eggleton
Fix a regression introduced in bfecbb091bde86a6df9980095379b31480dbb759. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-25dockersetup: fix error when printing URL with https enabledPaul Eggleton
Copy-paste error. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-25import_otherdistro: try-specfile: handle files in current dirPaul Eggleton
The try-specfile subcommand is for testing, so it's entirely possible the spec file we are testing with will be in the current directory; os.path.dirname() returns '' in this case, which os.path.relpath() then chokes on. Avoid this in the easiest way possible by just prepending ./ to the path in this case. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-25import_otherdistro: handle non-UTF8 encoded spec filesPaul Eggleton
It's not common, but if someone saves a spec file with a non-UTF8 encoding then the surrogate characters were preventing the data from being saved to the database (at least with MariaDB). Instead of assuming UTF8, try a few common encodings on the first 100 lines. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-25Optionally allow accounts without security questions to reset passwordPaul Eggleton
Add a SECURITY_QUESTIONS_REQUIRED setting that defaults to True, but if set to False then a user who has not set security questions will still be allowed to reset their password. This is convenient for the OE Layer index because there are a number of existing accounts, none of which will have security questions set. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-25Set string representation for UserProfile & SecurityQuestionAnswer modelsPaul Eggleton
Makes the admin pages a bit more useful for debugging. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-25Fix admin logout login button redirectionPaul Eggleton
If you clicked on logout in the admin site then clicked on the "Login" button and logged in, you were redirected to the logout page because we weren't detecting that the redirection should not be applied. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-25auth_views: minor indentation fixPaul Eggleton
No code changes. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17requirements.txt: update pillowPaul Eggleton
This is the only dependency we can update right now; the rest will cause problems given that the layer index needs python 2 support still. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17Fix a couple of Edit Profile form message bugsPaul Eggleton
* Tweak message when one or more (but not all) security question answers have been changed so that it's clear that all are required * Ensure success message gets shown upon saving Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17Tweak password reset security question handlingPaul Eggleton
* Require all three answers to be correct (previously only two had to be) * Give the user three attempts instead of just two Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17check_requirements.sh: more tweaksPaul Eggleton
* Use venv module instead of deprecated virtualenv command * Upgrade pip at the start so we don't get warnings about it Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>