| Age | Commit message (Collapse) | Author |
|---|
|
srtool: cumulative deployment features and fixes
High level new features:
* Publishing support to external/public databases
* Ability to label products as "active", "inactive", "under development"
Inactive (EOL) products appear but
* Do not affect status propagation
* Do not auto-create defects
Development product status is not exported to pubic database
* Extend NIST download range to 2002..2019
* Added MITRE downloads to provide RESERVED tracking
* Extended audit history tracking and meta-data
* Delete CVE records
* Ability to do "OR" searches (default is "AND")
Example: "CVE-2019-20095 OR CVE-2019-20096 OR CVE-2019-19977"
* Automated defect creation (Jira)
If selected, creates customer defect for selected and active products
Reuse existing defect if present for given product
* Many small sorting, readability, edge case fixes
Backups:
* Add meta-data stamp file for each backup
* Save daily backups with day name instead of day number
* Preserve file dates when making copies to backup
* Add list command
Automated Updates:
* Fix report format
* Add trial run test
Utilities:
* Add 13 new database fix up procedures
Some are one-shot historical fixes, some are learned validation checks
Database Schema:
* Add "SRTool" class to wrap shared enumerations (e.g. Priority)
* Add "Update" class to tag and track audit trail objects
* Change Priority naming to match CVE model instead of JIRA
* Add srt_created/srt_updated to CVE/Vul/Inv/Notify for improved updating and auditing
* Add to Defect the SRT versions of Status, Priority, Outcome
To distinguish these from the customer's defect system's values
Common Tools:
* Fix new CVE auto-scoring to skip CVE's already scored (though still NEW)
* Add automated propagation of Defects/Investigations status to parent Vulnerabilities
See "srtool_common.py" for rule details
CVEs:
* Add MITRE as an automatic upstream source
This is to specifically capture all of the "RESERVED" CVE enumerations which
will not appear in the MIST databases, and have the CVE records in place for
internal investigations and transitions to "public" status.
* Spell out the command arguments in the NIST data source files for greater legibility
* Change Priority naming to match CVE instead of JIRA
* Add parallel status states for "inactive" products
This specifically blocks state propagation from inactive objects to active objects
NIST management script:
* Refactor file for greater clarity
* Reorder methods to reflect workflow order
* Fully spell out names of objects
* Remove temporary holding class "CVE" in favor of dictionary objects
* Debugging enhancements
* Incremental update commands for stepped debugging
For example, ability to fetch/update specific CVE(s)
* Additional debugging flags
[YOCTO #13734]
Signed-off-by: David Reyna <David.Reyna@windriver.com>
|
|
Fix the pylint errors in the bin directory tools. The pylint command was:
$ PYTHONPATH=./lib:./bin pylint3 --load-plugins pylint_django bin --disable=C,R,unused-variable,unused-wildcard-import,redefined-outer-name,unused-argument,fixme,bare-except,broad-except,redefined-builtin,unnecessary-pass,logging-not-lazy,wildcard-import | tee srt_bin_pylint.txt
The currently allowed exceptions are:
W0603: Using the global statement (global-statement)
W0611: Unused ORM imported from srt_schema (unused-import)
Also, remove the obsolete 'bin/common/srtool_defect.py' and skip the need
to pylint it altogether.
Signed-off-by: David Reyna <David.Reyna@windriver.com>
|
|
Complete the support for backgroup data source updates:
* Add cron-start,cron-stop to srtool_update
* Have cron update run as a user space script to avoid sudo
* Hook cron-start,cron-stop into srt start,stop
* Add list command to show update sources
* Have force command propagate to update script calls, and
add force option to all source scripts
* Add 'srt manage update ...' for access to the update functions
* Add flag SRT_SKIP_AUTOUPDATE and srt option noautoupdate to
disable the automatic update app for development assistance
Related Fixes:
* Set the schema generator to always update on startup (13138)
* Fix CVE 'recommend' default to the integer zero (13139)
with auto-fix at startup for existing databases
[YOCTO #13131]
[YOCTO #13138]
[YOCTO #13139]
Signed-off-by: David Reyna <David.Reyna@windriver.com>
|
|
Fix the data source update mechanism:
* Move the update functions to "bin/common/srtool_update.py"
* Remove 'lastModifiedDate' from the data source JSON files (since
every restart overwrites any updated values)
* Change the 'update_time' field to a dictionary of offset values
e.g. "{\"weekday\":\"6\",\"hour\":\"2\"}" = day of week, hour of day
* Implement the update frequency calculations
* Implement data source name filters for selected manual updates
* Add a log status file
[YOCTO #13131]
Signed-off-by: David Reyna <David.Reyna@windriver.com>
|
|
|
|
Changes:
Repartition the data sources
Reconfigure the data sources into self-contained directories under the "bin" directory.
Implement dynamic data source discovery and import
Remove all hard coded data source data (e.g. fixtures, data, CVE lookups)
Add license files to all data sources
Django User model
Add "users" Django application dir
Login page
Self create user account page
Password change page
User access and delete management
CVE
Name sorting by hidden 'name_sort' field (CVE-nnnn-0nnnnnn)
CVE Triage
Auto import reserved CVEs
Add MITRE CVE records where NIST missing
Add data source count to triage page
Easy checkbox toggle by clicking any field
Triage any CVE status category (not just new)
Assign to any CVE status category
Object create/delete
Create/Delete Vulnerablities
Create/Delete Investigations from Vulnerablity page
Add "Historical" CVE status
When bootstraping system, all CVEs older than 60 days preset to "Historical"
Add CVEs withint 60 days preset to "New"
Can be overridden by defect and systaining status imports
Preadd Debian data for "New" CVEs
Abstraction
Add generic Product mappings to defect system ("defect_tag": defect prefix)
Add generic Product mappings to product system ("product_tag": product reference, related)
Manage functions via "srt" script
For example add superuser
Normalize Vulnerability to Investigation mapping
Replace orm_vulnerabilityproduct with orm_vulnerabilitytoinvestigation
General
Enable the 'srtool-requirements.txt' Django test
Speed the CVE scoring by pre-fetching the datasources
Progress display cleanup
Move and update srtool_defect prototype to 'bin/yp'
Signed-off-by: David Reyna <David.Reyna@windriver.com>
|
David Reyna
Ross Burton