aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)Author
2019-02-09srtool_investigation: list investigation defects and statusDavid Reyna
In the Investigation page, explicilty list the attached defects and their status (and controls) for clarity and simplicity. Signed-off-by: David Reyna <David.Reyna@windriver.com>
2019-02-05srtool_utils: add master app clone supportDavid Reyna
Update the master app tool to create new master apps. Also, update the master app change feature to remove the previous apps data source entries. Note that the existing users and products are left untouched (to keep the database records working); such such obsolete content must be removed manually or the data base restarted clean. Creation example, using 'yoyo' for the "Yoyodyne Corporation": $ ./stop.sh $ ./master_app create yoyo $ ./start.sh Switch bask to Yocto Project example: $ ./stop.sh $ ./master_app yp $ ./start.sh Signed-off-by: David Reyna <David.Reyna@windriver.com>
2019-02-03srtool_search: enhance search with OR, Quotes, ExcludeDavid Reyna
Enhance the search query processing: * By default, space separators are treated as AND * Add the explicit keywords OR and AND Note: these keywords switch the default separator * Allow OR searches in addition to AND searches Note: mixed searches use the SQL precedence rules * Allow a '-' prefix to exclude a string * Allow single and double quotes for exact matches Examples: * abd def (equals: abc AND def) * abc OR def (equals: OR abc def) * abc OR "def ghi" (equals: abc OR 'def ghi') * abc OR def AND -ghi Signed-off-by: David Reyna <David.Reyna@windriver.com>
2019-01-31srtool_cve: add multiple cve attach to a vulnerabilityDavid Reyna
Add these features: * Allow attaching multiple Vulnerabilities to a CVE * Allow attaching existing Vulnerability to a CVE * Allow attaching multiple triaged CVEs to a Vulnerability Revert change to to execute_process. Some routines that use this method already apply a "decode()", and you cannot do two decodes on an object. Signed-off-by: David Reyna <David.Reyna@windriver.com>
2019-01-31srtool-updates: improve update reportingDavid Reyna
Add time till next update in update verbose output. Indicate in table display when a datasource has an update policy but no update script. Signed-off-by: David Reyna <David.Reyna@windriver.com>
2019-01-30srtool-mitre: implement CVE NEW_RESERVED statusDavid Reyna
Update the Mitre scanning tool to convert recent (and/or no date) reserved CVEs to the state NEW_RESERVED. This will keep them separate from the new CVEs that need triage, plus keep them from being unnecessarily scored by the background process. Add a fixup rountine for previoulsy imported databases: $ ./bin/common/srtool_utils.py --fix-new-reserved Signed-off-by: David Reyna <David.Reyna@windriver.com>
2019-01-29srtool_email: update email values, error handlingDavid Reyna
Various updates and fixes: * Use the new SRT_EMAIL_* variable names * Fix hardcoded value for 'from' address * Add additional error handling * Allow the email settings to be defined in SrtSetting values, and provide example in the ACME datasource file Signed-off-by: David Reyna <David.Reyna@windriver.com>
2019-01-24srtool: separate name/passwords for defect and emailDavid Reyna
Separate the environment variables of the username and password for the defect and email systems into separate values, in case they need separate credentials. Also, fix a Wind River-ism in the Jira template to reflect that the product key is not necessarily also defect name prefix. Signed-off-by: David Reyna <David.Reyna@windriver.com>
2019-01-22srtool_README: update the git repo README fileDavid Reyna
Add the new web links and mailing list. Signed-off-by: David Reyna <David.Reyna@windriver.com>
2019-01-21srtool_start: generate SRTool stop/restart scriptsDavid Reyna
When the SRTool successfully starts, auto-generate helper scripts to allow the user to stop and restart the SRTool based on the previous options. This is also useful for scripts that need to stop then restart the SRTool server in order to safely perform actions (like backup or restore). Generated scripts: bin/srt_start.sh (also first stops the server if currently running) bin/srt_stop.sh Signed-off-by: David Reyna <David.Reyna@windriver.com>
2019-01-20srtool_backup: simplify the backup scriptDavid Reyna
Reset the backup tool to simply save the (a) sqlite database, the (b) data files, and (c) the attachments. Support both the weekly backup (to "backup_$year_$weeknum") and daily backup (to "backup_$weekday"). The previous JSON export format is being reworked. Signed-off-by: David Reyna <David.Reyna@windriver.com>
2019-01-19srtool_patcher: fix in-line change handling in public areaDavid Reyna
Only capture the custom sections in the in-line patch file, so that changes to the public areas are kept to the original file. This allows the user to push those public differences or remove them, and not affect the application of the custom patch file. Signed-off-by: David Reyna <David.Reyna@windriver.com>
2019-01-19srtool_patcher: add custom exclusion supportDavid Reyna
Add the ability to exclude sections of the common code. The primary use case is if the common code is executinga function that causes undesired side effects for the customization. Here is example code in "bin/acme/patcher/inplace/bin/srt.patch": ### ACME_EXTENSION_EXCLUDE_BEGIN ### #echo "The system will $CMD." # ### ACME_EXTENSION_EXCLUDE_END ### ### ACME_EXTENSION_BEGIN ### # # NOTE: Exclusions should be used only when necessary # NOTE: The excluded lines are commented so that they are inactive # but restorable # echo "The ACME SRTool system will now $CMD!" ### ACME_EXTENSION_END ### The content in the excluded section are commented, so that when it is cleaned those lines can be restored to the original. Signed-off-by: David Reyna <David.Reyna@windriver.com>
2019-01-19srtool: add in-place custom patching supportDavid Reyna
Add 'in-place' patch support for customizing organization patches on top of regular upstream SRTool files. An example is provided in "bin/acme/patcher/inplace/bin/srt", which adds a custom hello message. * To assert (merge) the custom version for runtime, run: $ ./bin/common/srtool_patcher.py -J bin/acme/patcher.json -I * To stash the customized version and replace with clean version (but keep any changes in the public sections for pushing upstream), run: $ ./bin/common/srtool_patcher.py -J bin/acme/patcher.json -i Signed-off-by: David Reyna <David.Reyna@windriver.com>
2019-01-17srtool: clean up patcher scriptDavid Reyna
Fix the description and usage examples in the header. Remove unused command. Signed-off-by: David Reyna <David.Reyna@windriver.com>
2019-01-17srtool: patcher, Jira template, Jira ACMEDavid Reyna
Add formal support for extending and customizing shared template code files. * The initial use case is a shared Jira integration file that partners can extend to their particular installation. A working example is provided in the ACME directory: upstream: "bin/common/srtool_jira_template.py" custom : "bin/acme/srtool_jira_acme.py" * The custom sections are blocked off with comment tags (e.g. ACME) ### ACME_EXTENSION_BEGIN ### ... ### ACME_EXTENSION_END ### * The './bin/common/srtool_patcher.py' tool provides the support, for example: To merge shared upstream code into the custom Jira script, run: $ ./bin/common/srtool_patcher.py -j bin/acme/patcher.json --merge-original To merge edits in script's common areas back to upstream, run: $ ./bin/common/srtool_patcher.py -j bin/acme/patcher.json --merge-custom Signed-off-by: David Reyna <David.Reyna@windriver.com>
2019-01-16srtool: pylint for the bin toolsDavid Reyna
Fix the pylint errors in the bin directory tools. The pylint command was: $ PYTHONPATH=./lib:./bin pylint3 --load-plugins pylint_django bin --disable=C,R,unused-variable,unused-wildcard-import,redefined-outer-name,unused-argument,fixme,bare-except,broad-except,redefined-builtin,unnecessary-pass,logging-not-lazy,wildcard-import | tee srt_bin_pylint.txt The currently allowed exceptions are: W0603: Using the global statement (global-statement) W0611: Unused ORM imported from srt_schema (unused-import) Also, remove the obsolete 'bin/common/srtool_defect.py' and skip the need to pylint it altogether. Signed-off-by: David Reyna <David.Reyna@windriver.com>
2019-01-15views: use subprocess directlyRoss Burton
2019-01-15srtool: separate lastUpdatedDate, enforce date formatDavid Reyna
* Add lastUpdatedDate to track when data source was updated * Leave lastModifedDate to track upstream dates * Introduce DataSource.DATETIME_FORMAT and ORM.DATASOURCE_DATETIME_FORMAT to enforce date formatting in the lib and bin code * Explicitly set 'nocache' for the data source page, so the refresh will always show the latest [YOCTO #13131] Signed-off-by: David Reyna <David.Reyna@windriver.com>
2019-01-14srtool: updates cleanup #2David Reyna
Clean up after pull from RBurton pylint updates * Protect against missing CVE lookup call * Protect against disabled defect tool when creating defects * Repair CVE 'score_date' data field default * Update tool typos for formatting Signed-off-by: David Reyna <David.Reyna@windriver.com>
2019-01-14orm: pylintRoss Burton
2019-01-14pylint srtmainRoss Burton
2019-01-14pylint srtguiRoss Burton
2019-01-14srtgui: pylint-inspired cleanupsRoss Burton
- Fix whitespace - Remove redundant imports - Remove redundant semicolons - Remove redundant try/except - Remove redundant super()
2019-01-14lib/orm: clean up importsRoss Burton
2019-01-14acme: cleanup importsRoss Burton
2019-01-14srtool-requirements: add requestsRoss Burton
This is used by lib/srtmain/management/commands/perf.py.
2019-01-14srtool: auto-update support cleanupDavid Reyna
Add backgroup data source updates cleanup * Add robust method for scanning ORM string lists * Fix DATASOURCE_FREQUENCY_STR ORM string list * Adjust '--list' columns [YOCTO #13131] Signed-off-by: David Reyna <David.Reyna@windriver.com>
2019-01-14srtool: separate update, backup, and utils scriptsDavid Reyna
The "srtool_utils.sh" script is separated into three scripts: * srtool_update.sh: database background updates * srtool_backup.sh: database backup tools * srtool_utils.sh: remaining utility actions The srtool_update.sh was pushed in a previous commit. Signed-off-by: David Reyna <David.Reyna@windriver.com>
2019-01-14srtool: complete auto-update supportDavid Reyna
Complete the support for backgroup data source updates: * Add cron-start,cron-stop to srtool_update * Have cron update run as a user space script to avoid sudo * Hook cron-start,cron-stop into srt start,stop * Add list command to show update sources * Have force command propagate to update script calls, and add force option to all source scripts * Add 'srt manage update ...' for access to the update functions * Add flag SRT_SKIP_AUTOUPDATE and srt option noautoupdate to disable the automatic update app for development assistance Related Fixes: * Set the schema generator to always update on startup (13138) * Fix CVE 'recommend' default to the integer zero (13139) with auto-fix at startup for existing databases [YOCTO #13131] [YOCTO #13138] [YOCTO #13139] Signed-off-by: David Reyna <David.Reyna@windriver.com>
2019-01-13srtool: fix core update implementationDavid Reyna
Fix the data source update mechanism: * Move the update functions to "bin/common/srtool_update.py" * Remove 'lastModifiedDate' from the data source JSON files (since every restart overwrites any updated values) * Change the 'update_time' field to a dictionary of offset values e.g. "{\"weekday\":\"6\",\"hour\":\"2\"}" = day of week, hour of day * Implement the update frequency calculations * Implement data source name filters for selected manual updates * Add a log status file [YOCTO #13131] Signed-off-by: David Reyna <David.Reyna@windriver.com>
2019-01-12srtool: add missing environment filesDavid Reyna
Add the default environment extension files for 'bin/common' and 'bin/yp'. They are currently passive. Signed-off-by: David Reyna <David.Reyna@windriver.com>
2019-01-12srtool: improve CVE status assignments, add CVE 2019David Reyna
1. Add the CVE 2019 data soures for MITRE and NIST. 2. Improve the CVE default status assignment system: * During the "Init" phase all CVEs default to HISTORICAL, unless they are within the CVE_INIT_NEW_DELTA date range. The value CVE_INIT_NEW_DELTA is defined in "bin/common/datasource.json", and is an out-of-box courtesy to provide some CVEs for triage in newly initialized systems. Changing the default value to '0' disabled this. * During the "Update" phase, CVEs default to NEW (and thus primed for triage) * Better separate the Init versus Update functions in "srtool_mitre.py" and "srtool_nist.py", and their respective datasource files. * Remove the post-process "preset_new()" in "srtool_common.py" in favor of directly computing the values in get_cve_default_status() in "srtool_mitre.py" and "srtool_nist.py", for speed and consistency. [YOCTO #13134] [YOCTO #13135] Signed-off-by: David Reyna <David.Reyna@windriver.com>
2019-01-08srtool: remove obsolete 'orm_cvereference' table checkDavid Reyna
Remove the obsolete and now empty 'orm_cvereference' table from the sanity check. Signed-off-by: David Reyna <David.Reyna@windriver.com>
2019-01-07srtool: fix parsing CVE reference separatorDavid Reyna
The initial implementation of passing CVE references used ';' as a separator. However, some URLs use this charater to include git branch information, for example: http://git.qemu-project.org/?p=qemu.git;a=commitdiff;h=4c65fed8b... Changing the separator characted to a tab fixes this and other unexpected characters. [YOCTO #13121] Signed-off-by: David Reyna <David.Reyna@windriver.com>
2019-01-07srtool: fix yp product order numberingDavid Reyna
Signed-off-by: David Reyna <David.Reyna@windriver.com>
2019-01-07yoto-project-products: add 2.7/WarriorRoss Burton
2019-01-07yocto-project-products: fix CPEsRoss Burton
The first number is the CPE version so this should always be 2.3. Fix the Yocto Project release version for Thud to be 2.6 instead of 2.5.
2019-01-07bin/*/srtool: don't try Python 2 importsRoss Burton
2019-01-07srtool: add 'superuser' helper script, improve 'tail'David Reyna
Add a devtool helper script 'suport.sh' to help start the super user setup call. Add 'srt_err.log' to 'tail.sh'. Signed-off-by: David Reyna <David.Reyna@windriver.com>
2019-01-07srtool: set priority/components new defect from investigationDavid Reyna
Add pulldown to set a new defect's priority and components in in the Investigation screen. Clean up the data passing from the srtool_defect* call. Signed-off-by: David Reyna <David.Reyna@windriver.com>
2019-01-04srttool: include missing CVE reference informationDavid Reyna
The CVE 'resource' and 'source' values for the CVE references are now scanned and displayed. * The JSON scanning has been moved away from CveResources to a dynamic value in the CveDetail record, similar to the CPE table processing. * Additional debugging support has been added * The now unused CveResources table will be deleted in a later revision [YOCTO #13121] Signed-off-by: David Reyna <David.Reyna@windriver.com>
2019-01-01srtool: parameterize the defect new callDavid Reyna
Change the new defect call to use named parameters. This will enhance the readability and better allow for future changes. Also, pass the CVE list and defect 'reason' so that the defect integation tool can use that for the defect record and/or use in creating its own version of the defect 'summary'. Signed-off-by: David Reyna <David.Reyna@windriver.com>
2018-12-30srtool: add defect create from investigationsDavid Reyna
Enable the feature of creating defects from investigations. Consolidate into one defect creation method for both investigations and CVE triage. Enhance the "srtool_defect.py" sample tools to simulate creating new defects. Fix the sample "srtool_jira.py" tool new defect creation to support the new "defect_tag" variable. Signed-off-by: David Reyna <David.Reyna@windriver.com>
2018-12-30srtool: replace 'toaster_render' with 'managedcontextprocessor'David Reyna
The 'toaster_render' was intended to define global context values. That feature is better provided by the existing 'managedcontextprocessor'. Signed-off-by: David Reyna <David.Reyna@windriver.com>
2018-12-30srtool: enable custom report extensionsDavid Reyna
Provide an example in ACME on how main apps can extent or replace existing reports. This example adds a new report "ACME Product Summary" type to the existing Product page export/report command. Also, fix defects in existing report.py. Signed-off-by: David Reyna <David.Reyna@windriver.com>
2018-12-30srtool: fix url patterns for master appDavid Reyna
Update the 'urlpatterns' processing to use the master app. Also, update the YP master app to include a url and view class, plus provide a default YP landing page, and abtract the default logo display. Signed-off-by: David Reyna <David.Reyna@windriver.com>
2018-12-30srtool: replace 'datasource_org' with new master app codeDavid Reyna
Transition the datasource scanning from 'datasource_org' to the new master app environment variable, so that it all works off of one key. Also, add a sample logo for ACME, plus fix datasource trace details. Signed-off-by: David Reyna <David.Reyna@windriver.com>
2018-12-29srtool: generalize the master app (yp, acme, ...) managementDavid Reyna
The SRTool allows users to substitute an alternate master application instead of the default "yp" in order to customize their instance to their organization. This is done by: (a) Creating a datasource directory under bin (b) Defining a "datasource.json" file (c) Defining 'export SRT_MAIN_APP="<app>"' in "srtool_env.sh" This environment files are scanned by 'bin/srt', and if such an alternate master app is found it pre-empts the default 'yp'. This value is set via the environment because "lib/srtmain/settings.py" is the file that sets the app (and this the URL) ordering, and it is processed before any database is attached. To disable the alternate main app, simply rename its "datasource.json" file and it will be ignored for the next start. The sample alternate app "acme" is provided to demonstrate this facility. Additionally, a development tool 'bin/dev_tools/master_app.sh' has been added to help switch between master apps, to aid testing. $ ./stop.sh $ ./master_app.sh acme $ ./start.sh ... test ... $ ./stop.sh $ ./master_app.sh yp $ ./start.sh Other included fixes: * Fix the ACME JSON files formating * Remove ACME "_sample" from all but "datasource.json_sample" * Fix tabs to spaces in "srt" * Add global contect values to views::managedcontextprocessor so that other app templates can share them Signed-off-by: David Reyna <David.Reyna@windriver.com>
2018-12-29srtool: add quick test for python3 and sqlite3David Reyna
Signed-off-by: David Reyna <David.Reyna@windriver.com>
generated by cgit 1.2.3-korg (git 2.39.0) at 2024-04-19 00:07:06 +0000