aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)Author
2018-09-11import_layer.py: add -t option for layer_typerbt/liRobert Yang
Now the logic is: Use options.layer_type if specified, and guess if not. Default to 'M'. Note choices=['A', 'B', 'S', 'D', 'M', ''], the '' is for default='', we can't use default='M' here, otherwise we don't know whether the 'M' is specified by user or is the default value, we don't guess if it is specified by user, otherwise, guess. Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
2018-09-06TODO: add some more itemsPaul Eggleton
There's always more work to do... Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-09-06rrs/tools/daily_run.sh: don't reload when updatingPaul Eggleton
There's no need to use --reload here every time. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-09-06rrs: admin: add in-line for Milestones to ReleasesPaul Eggleton
If you're creating a new release then you will need to create milestones within it, so add in-line forms for these. Unfortunately there's no capability yet to automatically split up the release into n milestones (or perhaps copy milestones from a previous release), that will have to wait until later - for now this makes things a little easier. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-09-06rrs_distros: match recipe on filename not PNPaul Eggleton
In the current RRS we should expect that more than one recipe with the same PN can exist in a layer - in fact it is common to have this (i.e. multiple versions of the same recipe). Use the filename to match the recipe record instead. At the same time, fix the exception handling. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-09-06rrs: show warnings for missing current release/milestonePaul Eggleton
If there is no release or milestone covering the current date, show an error message at the top of the recipes list page to alert the user that they can't view current data (since this is a common cause of not being able to see that which may not be immediately apparent). Additionally, show a warning within rrs_upgrade_history when this happens. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-09-06rrs: fix broken links when there are no milestones in a releasePaul Eggleton
If a release has no milestones, we shouldn't be selecting it as the default to be linked to in the maintenance plan drop-down, so filter those out. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-09-06rrs: fix recipe count not showing up properly if no per-recipe maintainersPaul Eggleton
If a maintenance plan has no per-recipe maintainers then we don't show the maintainer filtering drop-down, but we still had javascript code that unconditionally tried to access it and of course that failed if it wasn't there. Disable that as well without per-recipe maintainers. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-09-06rrs_distros: ensure we only run distro processing oncePaul Eggleton
We do not need to get the distro package lists for every layerbranch, just once. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-09-06rrs/tools: handle dry-run properlyPaul Eggleton
In the case of dry-run a couple of the scripts were breaking out after one layerbranch had been processed due to the code structure. Handle the exception within the block for the layerbranch to avoid this. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-09-06reviewdetail: show newlines in descriptionPaul Eggleton
I already changed the main layer detail page in 3adddf6a25977cb576f8fcefd307d947e43d637b to show newlines, but forgot to do the same in the review detail until now. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-09-06Limit selection of "Base" layer typePaul Eggleton
If a user doesn't have publish rights and the type of the layer isn't already "Base" then disallow selecting the Base layer type. Some submitters are selecting this type for their own layers, but it's pretty much reserved for openembedded-core and meta-oe (so that they appear at the top of the layer list). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-09-06Disable git interactive password promptsPaul Eggleton
We do not want to be prompting the user for a password during layer updates or upstream checks, e.g. in the case where a repo requires authentication, or on github where any fetch of a nonexistent repo apparently triggers authentication. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-09-06update: do not try to parse a layer that failed to fetchPaul Eggleton
If a layer's repository didn't fetch properly then we should not be trying to parse it. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-08-29docker: correct comment in .gitconfigPaul Eggleton
We moved this when we switched to using a specific user account. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-08-29docker: remove one step from proxy setupPaul Eggleton
We can always deploy these files since the default versions have all the settings commented out - save proxy users from needing to uncomment these (it's annoying if you miss doing so). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-08-24update: add option to stop immediately on parse errorsPaul Eggleton
During debugging a parsing issue, we don't really want to continue if a parsing error occurs, and in that situation I usually end up using Ctrl+C to exit early. Add an option to exit immediately upon error to avoid having to do that. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-08-22Hide navbar on comparison select pagePaul Eggleton
We don't want to see the navbar on this page because otherwise it lets you select branches and click on Stats etc. which doesn't make any sense in this context. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-08-22rrs: fix recipe/maintainer count showing at topPaul Eggleton
We were using the main badge class as a selector here to show the recipe or maintainer count, and that meant that it also showed up at the top next to the login button if there were some layers to review. Use a proper id to stop that from happening. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-08-14Include errors/warnings in main log in error/warning counts for updatesPaul Eggleton
When showing the error/warning counts for update records we need to include any errors/warnings that are shown only in the main update log, so we need to adjust how these are collected. Use a function rather than pure aggregation to give a bit more control, and a {% with ... %} block in the template to avoid the functions being called more than necessary. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-08-14update: ensure that exceptions during the update process are loggedPaul Eggleton
If an exception occurred during the update then we were managing to save the update record, but we did not include the exception traceback in the log for the update. Catch the exception and log it which ensures it gets captured in the update record and still gets printed as well. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-08-14Improve collection/display of LayerUpdate recordsPaul Eggleton
Make layerupdate collection slightly more reliable and make it easier to see when updates have actually been captured: * Split layerbranch into separate layer and branch fields, since there may not be a layerbranch in existence but we might want to log an error relating to the branch and layer. * Show all layerupdates on the update detail page, not just those with log messages * Record before and after revisions and show these in the update detail and layerupdate detail (with links) * Record return code of update_layer process * Highlight layer updates with a non-zero return code, errors or warnings in the output on the update detail page * Show duration on the layerupdate detail page Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-08-14update_layer: improve exception handlingPaul Eggleton
When an exception occurs during the main part of update_layer, we were catching and printing it but that's not enough - we need to do the following as well: * Use logger.error() to print the exception information, so that it gets logged and highlighted as an error in the layer update * Exit with a non-zero return code so that update.py knows it has failed Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-08-14update: avoid errors if values could not be found in initial outputPaul Eggleton
If for some reason update_layer.py does not print out the values we expect then we shouldn't be throwing a traceback, we should be handling it gracefully - i.e., print an error and then move on to the next layer. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-08-13Show links to include files on recipe detailPaul Eggleton
On the recipe detail page we provide a link to the actual recipe file for reference purposes. However, for recipes that include a common .inc file, many of the definitions of variable values will be inside the .inc, therefore if you just look at the recipe you won't see the full picture. Of course you can just go up to the parent directory in the repository web interface, but for convenience's sake add links to any files that are included/required by the recipe that are adjacent to the recipe itself. (We already have the data in the form of the RecipeFileDependency records that are intended to ensure we know when the recipe needs to be updated if one of the files it includes changes). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-08-13Add side-by-side comparison detail and enhanced selectionPaul Eggleton
Specifying the covering recipe in the comparison recipe detail page was always a bit awkward - you could only type the name, if you wanted to actually find a recipe or look up the currently selected one's details then you had to open another browser tab/window. To fix this, replace the form on the comparison recipe detail page with a side-by-side display of the covering recipe's information, along with a button that lets you search and then select the covering recipe and at the same time enter comments or set any of the other cover fields. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-08-13README: add setup info for other distro comparisonsPaul Eggleton
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-08-13Show source URLs on recipe detail pagePaul Eggleton
If we're collecting this useful info we should really display it. Try to make the link clickable if possible. At the same time, add the layerbranch to the list display for Source objects in the admin site. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-08-13Add script to fetch Fedora package sourcesPaul Eggleton
Add a script to fetch package sources from Fedora's Pagure infrastructure site at https://src.fedoraproject.org/, based upon github-fetch.py (I broke out the internal fetch function with the intent to share it with that other script, however I didn't in the end as I wanted the scripts to remain independent). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-08-13Add script to fetch all repos under a github organisationPaul Eggleton
Add a script to fetch/update all repositories within a GitHub organisation. Note that to run this you will need an access token, which you can create by going to GitHub and then Settings -> Developer Settings -> Personal access tokens. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-08-13Add ability to trigger comparison updates manually from UIPaul Eggleton
Comparison updates might involve some custom fetch process, so provide a mechanism to register these via settings.py on a per-branch basis. If an update command is defined for a branch and the logged-in user has the new "update_comparison_branch" permission, an "Update" button will show up on the recipes page for the comparison branch for authenticated users that will trigger the command in the background (as a celery job) and then show a page that displays the status. The status isn't shown in real-time since that requires quite a lot of plumbing, but the page at least auto-refreshes. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-08-13Fix comparison layers showing up in dependencies list in edit layer pagePaul Eggleton
We weren't filtering on the right flag here when editing (but we were when submitting). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-08-13Rename LayerItem classic field to comparisonPaul Eggleton
Make it clearer what this field is for (it should be set for every comparison layer, so that they don't show up in places they shouldn't). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-08-13rrs: show link to other distro comparison pagesPaul Eggleton
If other distro comparison package(s) exist for a recipe being shown in the RRS recipe detail page, link to the page for each package as well as any extra URLs. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-08-13Add ability to store extra URLs to be displayed for comparison recipesPaul Eggleton
Add a structure that lets you define a template URL per layer to be shown per comparison recipe. For example, you could use this to define a URL template to link to the upstream summary page for the package (e.g. Fedora's page for the acl package is at https://apps.fedoraproject.org/packages/acl, so you would use https://apps.fedoraproject.org/packages/%pn% as the template and then this would be shown for every package). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-08-13Add basic CSV export for other distro comparisonsPaul Eggleton
Add the ability to export the comparison search results to CSV format in order to allow importing the data into external tools. Note: I implemented this in a different way than the earlier recipe CSV export, i.e. it uses a template to render the CSV instead of a function-based view with the Python csv module - the reason for this is we can reuse the same view as we use for producing the search, with all of the flexibility that gives us. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-08-13Support excluding inherited classes from other distro comparison reversed queryPaul Eggleton
When doing the reversed query it's often desirable to exclude recipes by inherited class, for example those that inherit the packagegroup, image and meta classes as they don't actually build anything and thus aren't going to match up with anything in the other distribution. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-08-13Support selecting more than one layer in other distro comparison search formPaul Eggleton
Provide a lazy-loaded popup for selecting layers to include in the query instead of having it as a simple drop-down, so you can select more than one layer. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-08-13Support reversed other distro recipe list / filteringPaul Eggleton
Enable "reversing" the comparison, so you can see which recipes on the OE side match up (or don't) with the other distro. The filtering for this is a bit awkward, since we don't have an actual foreign key for the link, hence the hairiness of the code. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-08-13Add support for importing Debian package infoPaul Eggleton
Add support for importing package information extracted from a running Debian system (i.e. from the output of apt-cache show "*"). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-08-13Add "needs attention" flag for comparison recipesPaul Eggleton
Add a flag that can be set and searched for to indicate that we need to take care of importing a package or a patch applied by a package. Ideally the comments would elaborate on what's needed (if it's not obvious from the cover status). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-08-13views: fix unknown categories comparison stats graph to include "Not available"Paul Eggleton
Packages with the "Not available" status were inexplicably excluded from the graph, include them. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-08-13Enhance filtering comparisonsPaul Eggleton
Allow searching on: * Any "available" status (i.e. other than "Unknown" or "Not available") * Whether the package has patches or not * What the covering layer is (assuming there is one). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-08-13Allow searching for uncategorised recipes in the comparison searchPaul Eggleton
It's a bit crude, but accept '' or "" as meaning search for entries with an empty category. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-08-13update_classic_status: add skip optionPaul Eggleton
Add an option to skip certain recipes by name (a crude workaround for when we would otherwise get an erroneous match). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-08-13update_classic_status: look for certain prefixes in order to matchPaul Eggleton
Some typical package name prefixes can be recognised and used to match. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-08-13update_classic_status: use source URLs to match python/perl recipesPaul Eggleton
If the source URL is fetching from pypi then we know it's python; similarly cpan means perl. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-08-13update_classic_status: set cover_pn from pn for direct matchPaul Eggleton
If we always set cover_pn if there is a match rather than only setting it if it's different from pn then it simplifies the code elsewhere. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-08-13update_classic_status: update for other distro comparisonsPaul Eggleton
* Only apply OE-Classic-specific logic after checking that it's actually OE-Classic * Ignore recipes marked deleted Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-08-13Add script to import from other distrosPaul Eggleton
Add a script to import "recipe" information from other distro metadata, based on the import_classic.py script. At the moment, this assumes a directory where each subdirectory is a package directory containing a spec file; this would be suitable for distributions such as Fedora assuming you have all of the package repos checked out locally. Since you can add additional information to these records (the cover fields pointing to matching recipes), existing records are updated rather than deleting everything and re-importing, and we only mark records as deleted rather than actually deleting them (in case you accidentally point the script at an empty directory or similar). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>