aboutsummaryrefslogtreecommitdiffstats
path: root/layerindex
AgeCommit message (Collapse)Author
2024-02-06layerindex: fix edit layer vcs_subdir max lengthTim Orling
EditLayerForm was missed in the previous commit. [YOCTO #15374] Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2024-02-05layerindex: allow periods in layer nameTim Orling
There is nothing preventing a period in a layer name from the BitBake perspective. For example, https://github.com/webosose/meta-webosose has: meta-webos-backports/meta-webos-backports-4.1 [YOCTO #15373] Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2024-02-05layerindex: allow Repository subdir to be 60 charsTim Orling
https://github.com/webosose/meta-webosose has layers like: meta-webos-backports/meta-webos-backports-4.1 which is 45 characters. Allow for up to 60 characters in the vcs_subdir field in the LayerBranch class. [YOCTO #15374] Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2024-01-30layerindex: allow empty Actual branchTim Orling
Exposing the actual_branch field of LayerBranch in the UI was intending to _enable_ setting a non-default value, but the check was accidentally left in a state that _requires_ a value. [YOCTO #15376] Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2024-01-22layerindex/update_layer.py: enable classes-global,-recipeTim Orling
Add support for the new BBClassGlobal and BBClassRecipe sub-classes. [YOCTO #15238] Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2024-01-22layerindex/views: add classes-recipe,-globalTim Orling
Since commits: bitbake f33ce7e7 'BBHandler/cooker: Implement recipe and global classes' oe-core f5c12800 'classes: Update classes to match new bitbake class scope functionality' we now have a lot of the oe-core recipe classes in meta/classes-recipe. We are also missing any bbclasses in meta/classes-global [YOCTO #15238] Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2024-01-22layerindex/recipeparse.py: extend bbclass regexTim Orling
Extend the bbclass regex to match classes-global and classes-recipe [YOCTO #15238] Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2024-01-22layerindex/models: add BBClassRecipe BBClassGlobalTim Orling
Add support for classes-global and classes-recipe, but use "proxy=True" to not create new tables for the new classes. Rather, we use the bbclass_type field. [YOCTO #15238] Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2024-01-22global: deprecated pkg_resources parse_versionTim Orling
Since we are only using parse_version for comparison (typically checking that we are greater than some minimum version for tool or package), one would think we can use packaging.version.parse as if it was parse_version Unfortunately, this requires conforming to PEP-440 version definitions, which does not work for e.g. autotools (2.72d) nor older openssl (1.1.1p). We rely in these (and to be sure other) cases on the LegacyVersion behavior. https://packaging.python.org/en/latest/specifications/version-specifiers/#summary-of-differences-from-pkg-resources-parse-version "This specification purposely restricts the syntax which constitutes a valid version while pkg_resources.parse_version attempts to provide some meaning from any arbitrary string." In order to have the least impact to the overall code, we instead add packaging_legacy to requirements.txt and use packaging_legacy.version.parse as if it was parse_version. https://pypi.org/project/packaging-legacy/ https://github.com/pypa/packaging/pull/407 Since pypi.org itself is depending on packaging_legacy (in fact, a pypi dev developed the package), we can expect it to be supported for quite some time. https://github.com/pypi/warehouse/pull/13500 [YOCTO #15348] Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2024-01-03layerindex: improve updates for actual_branchTim Orling
Some layers now have one branch with many supported LAYERSERIES_COMPAT. If this branch name does not match one of the stable releases, LayerBranches might not have been created. When actual_branch is set, it is only set in a LayerBranch object. We previously could not update (create) a stable branch with actual_branch except manually in the admin interface. Add --force-create option to be used in conjunction with --actual-branch (which already requires --branch) in the update.py script. This tells the script to ignore the fact that no layerbranch exists already. Add --actual-branch to update_layer.py so that we can create (and more importantly checkout) an actual_branch for the given stable --branch. Update utils.py to allow checking out of actual_branch when a LayerBranch does not yet exist. While we are at it, ensure that any Branch that is marked as no update will be skipped even with --force-create. The main reason that a Branch has updates disabled is because the bitbake or python syntax has changed enough to cause exceptions. This script can now be run with: ./layerindex/update.py \ --layer meta-weird-one \ --branch kirkstone \ --actual-branch=nonstandard \ --force-create Which will attempt to create a meta-weird-one:kirkstone layerbranch checked out at the 'nonstandard' branch from that layer's git repo. This allows layerindex admins to at least populate the database without tedious creation of layerbranches in the admin interface. Helps make the "branch mapping" actually work and be useful: [YOCTO #8008] Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2024-01-03layerindex: add Update Layer UI featureTim Orling
* Add an "Update Layer" button to the layer detail view. - This allows a user that is a member of is_staff to trigger an update of the current layer (for the current branch) * Add an "Update Layer" button to the reviewdetail view - This allows a user that is a member of is_staff and has publish_layer permissions to trigger an update attempt of the layer under review (even in the un-published state) * The update is run as a task with Celery NOTE: You must have the RABBITMQ_ and DATABASE_ credentials set correctly in the docker/settings.py file or set via environment variables or you will get authentication errors talking to layersdb or layersrabbit containers. [YOCTO #12484] layerindex/views.py: add update_layer_view layerindex/urls.py: add update_layer_view layerindex/urls_branch.py: add update_layer_view templates/layerindex/reviewdetail.html: add Update Layer button templates/layerindex/detail.html: add Update Layer button templates/layerindex: add updatelayer.html TODO: While the update is happening, the AJAX rendering of the update.log is showing the b'' characters and not adding any new lines. If you go back to the same task view afterwards, the log is rendered as expected. TODO: After the update is completed, it would be nice to have a button to return you to the page from where you called the "Update Layer". Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2024-01-03layerindex: Add actual_branch to forms and viewsTim Orling
For layers which do not follow standard branch names (including the inclusive naming move away from "master" to "main") we have the actual_branch field set in a LayerBranch object. Previously this was only exposed via the admin interface. Allow layer maintainers (including upon submitting a new layer) to set the 'Actual branch' in the web UI. Add a check to make sure the actual_branch is a valid branch name using 'git check-ref-format --branch <actual_branch>' since we are not using full refs. [YOCTO #8008] NOTE: Only existing LayerBranches will be editable. A new layer can be submitted with a different branch for "master", but only the "master" LayerBranch will be created. Further changes to the update.py script will be needed to make creation of new stable branches with an actual_branch possible. Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2024-01-03layerindex/views.py: fix parse_view typoTim Orling
The Branch Comparison view would throw an error because of a parser_view instead of parse_view typo. [YOCTO #15332] Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-11-22utils.py: fix unclosed (Tim Orling
File "/opt/layerindex/layerindex/update.py", line 525 failed_layers[branch].append('%s: Failed to add since LAYERDEPENDS [%s ...] is not SyntaxError: '(' was never closed Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-11-17update.py: display first unsatisfied LAYERDEPENDSTim Orling
Make errors like this more useful: ERROR: Issues found on branch nanbield: meta-luneos: Failed to add since LAYERDEPENDS is not satisfied meta-luneui: Failed to add since LAYERDEPENDS is not satisfied In this case, meta-luneos depends on meta-luneui, but we cannot create a 'nanbield' layer branch, because meta-luneui LAYERDEPENDS on qt6-layer collection, which currently has no 'nanbield' branch. Use next(iter(value['deps'])) to get the first element in the OrderedDict Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-11-17update.py: show branch if missing conf/layer.confTim Orling
Currently, the error message is something like: ERROR: conf/layer.conf not found for layer meta-doom - is subdirectory set correctly? This is because in this case meta-doom has a 'langdale' branch without a conf/layer.conf Another example is: ERROR: conf/layer.conf not found for layer meta-st-stm32mp - is subdirectory set correctly? This is because meta-st-stm32mp has a master branch, but it only contains a README.md telling you to use the stable branches (e.g. nanbield). Make it more obvious what the source of the error is by also displaying the branch that was being attempted. Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-11-17layerindex/models.py: fix Inactive-Upstream checkTim Orling
The regex for Upstream-Status had (\w+), but this does not include hyphen/dash. Explicitly add it to the pattern. Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-11-16utils.py: get dependent layers at same branchTim Orling
For both meta-poky/meta-yocto and meta-xilinx/meta-xilinx-core we have a situation where the "collections" stayed the same ("yocto" and "xilinx" respectively) but the layer/layerbranch changed. Without the "branch" argument to get_dependency_layers, we were always defaulting to the older layer which first defined the "collection". Instead, add an option to use "branch" to filter on the expected LayerBranch object. Keep the old behavior just in case someone depends upon it. [YOCTO #15221] Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-11-16update_layer.py: checkout dep layers as wellTim Orling
We have never been checking out dependent layers at the same release/branch. With the introduction of 'addpylib', this became obvious due to parsing errors. Ensure that known LayerDependency objects are checked out at the expected branch/release. Since openembedded-core has already been handled elsewhere, we skip it. [YOCTO #15236] Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-11-16update.py: fix updates_enabled for newbranchTim Orling
When a new release is run for the first time, no LayerBranch exists yet and the update_enabled cannot be queried. The unintended result is that all layers are skipped. Move the update_enabled check into the code path where the layerbranch already exists. Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-11-16layerindex/tools: add mark_yp_compatible_layers.pyTim Orling
Add a script which can either mark one --layer --branch or use data in a --from-file to mark multiple layer:branch objects as Yocto Project Compatible. The --from-file is a json file is compatible with or which can be generated by: yocto-autobuilder-helper/scripts/list-yp-compatible-layers.py [YOCTO #15093] Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-10-19bootstrap: fix typo in minified file nameTim Orling
Fixes dropdown menus so they are functional again. Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-10-19update.py: skip unless layerbranch.updates_enabledTim Orling
If layerbranch.updates_enabled is false, skip the update. Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-10-19models.py: add updates_enabled to LayerBranchTim Orling
Especially since LTS branches have become more popular as the only active branch, we need to be able to turn off updates on a LayerBranch basis rather than the current Layer or Branch heavier hammers. Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-10-16Chart.js: upgrade from 2.9.3 to 2.9.4Tim Orling
https://github.com/chartjs/Chart.js/releases/tag/v2.9.4 CVE: CVE-2020-7746 https://nvd.nist.gov/vuln/detail/CVE-2020-7746 Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-10-16Chart.js: add version to filenamesTim Orling
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-10-16bootstrap: add version to filenamesTim Orling
To make it easier to quickly see what version we are using, add the version to the filenames. Also, use minified flavor in templates/base.html Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-10-16jquery: upgrade 3.4.1 -> 3.7.1Tim Orling
Use minified version in templates/base.html Fixes some vulernabilities: https://nvd.nist.gov/vuln/search/results?adv_search=true&isCpeNameSearch=true&query=cpe%3A2.3%3Aa%3Ajquery%3Ajquery%3A3.4.1%3A*%3A*%3A*%3A*%3Anode.js%3A*%3A* CVE: CVE-2020-23064 CVE: CVE-2020-11022 CVE: CVE-2020-11023 For full changelog see: https://github.com/jquery/jquery/compare/3.4.1...3.7.1 License-Update: Copyright OpenJS Foundation (from JS Foundation) Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-10-05Refactor usage of django.conf.urlsTim Orling
django.conf.urls.url() was removed in Django 4.0: https://docs.djangoproject.com/en/4.2/releases/4.0/#features-removed-in-4-0 Replace all usage with django.urls.re_path() Replace all django.conf.urls imports with equivalent django.urls modules Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-09-29layerindex/urls.py: fix submit/thanks/ urlTim Orling
The url returned by the reverse lookup ends in /submit/thanks/ but the regex was improperly searching for a string without the terminating "/". Thank you to Pawel Zalewski for reporting the 404 error. Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-12-20layerindex/utils.py: Add to baseconfig=True for bb.parse.handle()Robert Yang
Bitbake's api has been changed via: afb8478d3 parse: Add support for addpylib conf file directive and BB_GLOBAL_PYMODULES The conf file won't be parsed without baseconfig=True: bb.parse.ParseError: ParseError at /path/to/oe-core/meta/conf/layer.conf:132: unparsed line: 'addpylib ${LAYERDIR}/lib oe' Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-12-16global: replace deprecated distutilsTim Orling
Replace distutils.version.LooseVersion with pkg_resource.parse_version https://majornetwork.net/2021/05/comparing-version-numbers-in-python/ [YOCTO #14990] Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-12-16utils.py: Fix for is_commit_ancestor()Robert Yang
The runcmd() would print an "ERROR" on failure which causes confusion since the failure is expected on old branches, so subprocess.getstatusoutput to fix the problem. Minor rewording. Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-12-16admin.py: Remove has_delete_permission()Robert Yang
The following commit has added on_delete=models.CASCADE to the ForeignKey: === commit 2d526f9b0d363f3e442abc97ef9bd9fab37ee5e4 Author: Amber Elliot <amber.n.elliot@intel.com> Date: Thu Jun 25 15:57:34 2020 -0700 Updating models and imports for Django 2.2 upgrade. === The on_delete=models.CASCADE will remove the objects which reference the removed objects, so there will be errors when removing a LayerBranch with admin permissions: === Cannot delete layer branch Deleting the selected layer branch would result in deleting related objects, but your account doesn't have permission to delete the following types of objects: Append distro === This is because has_delete_permission() always return False, remove it to use Django's implementation to fix the problem. Minor rewording. Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-12-16utils.py: Use get_layer_var() to get BBFILE_COLLECTIONSRobert Yang
The commit 38e6288c7dad579518b0de2b6664f23be58889b6 has added oe-core to BBLAYERS by default, and then there will be multiple BBFILE_COLLECTIONS which causes update.py failure to update layers (most of the dependencies or recommends are missing), use get_layer_var() to get BBFILE_COLLECTIONS to fix the problem since it can ignore the extra oe-core from BBFILE_COLLECTIONS. Fix minor typos. Minor rewording. Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-04-28layerindex/utils.py: ignore 'core' in BBFILES_COLLECTIONSTim Orling
Many layers append BBFILE_COLLECTIONS and therefore have 'core <layer>' During update.py, this means we are likely not handling the collection we expect: WARNING: /opt/workdir/git___git_openembedded_org_meta-openembedded/meta-oe: multiple collections found, handling first one (core) only BBFILE_COLLECTIONS = "core" Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-04-28recipe{desc,parse}.py: BB_ENV_PASSTHROUGH_ADDITIONSTim Orling
ERROR: Variable BB_ENV_EXTRAWHITE has been renamed to BB_ENV_PASSTHROUGH_ADDITIONS ERROR: Variable BB_ENV_EXTRAWHITE from the shell environment has been renamed to BB_ENV_PASSTHROUGH_ADDITIONS ERROR: Exiting to allow enviroment variables to be corrected Replace BB_ENV_EXTRAWHITE with new variable BB_ENV_PASSTHROUGH_ADDITIONS In order to be backward compatible with older branches, we must first check for the presence of the bitbake commit which implemented the variable name change, using layerindex.utils.is_commit_ancestor(). Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-04-28layerindex/utils.py: add is_commit_ancestor checkTim Orling
Add a helper function to check if a given SHA1 hash is an ancestor in the currently checked out branch, using: git merge-branch --is-ancestor <commit> HEAD NOTE: This will not match commits which have been cherry-picked. Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-04-18layerindex/models.py: add Inactive-UpstreamTim Orling
Add the newish Inactive-Upstream upstream status. Add 0046_alter_patch_status.py migration. Signed-off-by: Tim Orling <tim.orling@konsulko.com> layerindex/migrations: update patch status Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-04-18layerindex/urls.py: fix about url patternTim Orling
The url pattern was not including the trailing / [YOCTO #14445] Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-01-13Updates for Django 3.0Tim Orling
* Update requirements.txt versions - Mostly update to latest pre-Django 3.1 versions * Fix deprecated axes.backends.AxesModelBackend - settings.py - docker/settings.py * Fix template syntax 'staticfiles' -> 'static' - base.html * Add migrations for layerbranch classicrecipe Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-01-13layerindex/*: make all shebangs python3Tim Orling
Several scripts still had /usr/bin/env python Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2021-10-20Report charset for text & CSV viewspaule/newfixes1Paul Eggleton
I noticed in the branch comparison plain text view ("Plain text" button in the Tools -> Branch Comparison page) that in current versions of Firefox the ellipses were coming through corrupted, though they looked fine in the HTML version, and it turns out this is because I wasn't specifying a character set encoding. It should be UTF-8, so add a charset to the content type stating as such for this and other similar views. Signed-off-by: Paul Eggleton <paul.eggleton@linux.microsoft.com>
2021-10-20utils: ensure we show error messages on server startupPaul Eggleton
If a BBHandledException occurs that means some error was logged, so we need to handle any pending events so that we can actually have the error logged. Tinfoil should really be doing this for us but at this stage in the release we can't really fix this there, so do it here for now. Signed-off-by: Paul Eggleton <paul.eggleton@linux.microsoft.com>
2021-10-20update: fix setup of BBLAYERSPaul Eggleton
So with honister / current master we can no longer get away with bypassing BBLAYERS - it now needs to point to the core layer at minimum. This is fine, we just need to skip parsing layer.conf if we're parsing the core layer or we get some extra warnings we don't need. Signed-off-by: Paul Eggleton <paul.eggleton@linux.microsoft.com>
2021-04-21Add SPDX license headers to layerindex-web source filesMeh Mbeh Ida Delphine
Added SPDX identifiers to all .py files except those in migrations directory. Fixes: [YOCTO #13527] Signed-off-by: Meh Mbeh Ida Delphine <idadelm@gmail.com> Signed-off-by: Paul Eggleton <bluelightning@bluelightning.org>
2020-06-30Replacing is_authenticated() with is_authenticated for Django upgrade.Amber Elliot
Signed-off-by: Amber Elliot <amber.n.elliot@intel.com> Signed-off-by: Paul Eggleton <bluelightning@bluelightning.org>
2020-06-30Updating models and imports for Django 2.2 upgrade.Amber Elliot
Removed all obsolete references to django.core.urlresolvers. Added the newly required on_delete fields to foreign key relationships in models.py and in all migrations. Signed-off-by: Amber Elliot <amber.n.elliot@intel.com> Signed-off-by: Paul Eggleton <bluelightning@bluelightning.org>
2020-01-06Fix parsing after AVAILABLE_LICENSES changePaul Eggleton
In OE-Core commit 8c9ef587fe499c612a878a1ab42092eb79b334ef an AVAILABLE_LICENSES variable was added in license.bbclass where its value is assigned with immediate expansion ( := ) with the result that it looks in LICENSE_PATH for licenses; in turn LICENSE_PATH refers to COREBASE. You might assume that COREBASE is always set, and normally it is (since it's set OE-Core's layer.conf) - but in the layer index context we do not parse layer.conf until a bit later, so it immediately fails. The quick way to fix this is just to set our own (correct) value for COREBASE and then AVAILABLE_LICENSES can be expanded successfully. You might ask why we don't instead just set BBLAYERS such that we *do* parse OE-Core's layer.conf - the answer is that that can have other effects such as BBFILE_COLLECTIONS being set, and at least at the moment the rest of the code isn't expecting that. Fixes [YOCTO #13723]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-12-09tasks: squash out CRs in task logs to avoid huge transferspaule/fixes15Paul Eggleton
Reloading an existing update task page was taking an extremely long time to fetch down the task log and then pegging the client CPU such that the browser gave a warning. Digging into it, logs from a Clear Linux update task can be of the order of 500MB in size (with all of the line refreshing using CRs that happens during downloads), causing (a) the transfer to take a long time and (b) the JS code that updates the log text box to be extremely busy. If we're loading the entire log from scratch (as we are when we refresh the page) rather than just getting an update since the last poll, we don't need any of those line refreshes - so squash them out before returning the data. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
generated by cgit 1.2.3-korg (git 2.39.0) at 2024-05-01 12:04:40 +0000