Age | Commit message (Collapse) | Author |
|
Fix the data source update mechanism:
* Move the update functions to "bin/common/srtool_update.py"
* Remove 'lastModifiedDate' from the data source JSON files (since
every restart overwrites any updated values)
* Change the 'update_time' field to a dictionary of offset values
e.g. "{\"weekday\":\"6\",\"hour\":\"2\"}" = day of week, hour of day
* Implement the update frequency calculations
* Implement data source name filters for selected manual updates
* Add a log status file
[YOCTO #13131]
Signed-off-by: David Reyna <David.Reyna@windriver.com>
|
|
Add the default environment extension files for 'bin/common'
and 'bin/yp'. They are currently passive.
Signed-off-by: David Reyna <David.Reyna@windriver.com>
|
|
1. Add the CVE 2019 data soures for MITRE and NIST.
2. Improve the CVE default status assignment system:
* During the "Init" phase all CVEs default to HISTORICAL, unless they are
within the CVE_INIT_NEW_DELTA date range. The value CVE_INIT_NEW_DELTA is
defined in "bin/common/datasource.json", and is an out-of-box courtesy
to provide some CVEs for triage in newly initialized systems. Changing
the default value to '0' disabled this.
* During the "Update" phase, CVEs default to NEW (and thus primed for
triage)
* Better separate the Init versus Update functions in "srtool_mitre.py" and
"srtool_nist.py", and their respective datasource files.
* Remove the post-process "preset_new()" in "srtool_common.py" in favor
of directly computing the values in get_cve_default_status() in
"srtool_mitre.py" and "srtool_nist.py", for speed and consistency.
[YOCTO #13134]
[YOCTO #13135]
Signed-off-by: David Reyna <David.Reyna@windriver.com>
|
|
Remove the obsolete and now empty 'orm_cvereference' table
from the sanity check.
Signed-off-by: David Reyna <David.Reyna@windriver.com>
|
|
The initial implementation of passing CVE references used ';' as a separator.
However, some URLs use this charater to include git branch information, for
example:
http://git.qemu-project.org/?p=qemu.git;a=commitdiff;h=4c65fed8b...
Changing the separator characted to a tab fixes this and other unexpected
characters.
[YOCTO #13121]
Signed-off-by: David Reyna <David.Reyna@windriver.com>
|
|
Signed-off-by: David Reyna <David.Reyna@windriver.com>
|
|
|
|
The first number is the CPE version so this should always be 2.3.
Fix the Yocto Project release version for Thud to be 2.6 instead of 2.5.
|
|
|
|
Add a devtool helper script 'suport.sh' to help start the super user
setup call. Add 'srt_err.log' to 'tail.sh'.
Signed-off-by: David Reyna <David.Reyna@windriver.com>
|
|
The CVE 'resource' and 'source' values for the CVE references are now
scanned and displayed.
* The JSON scanning has been moved away from CveResources to a dynamic
value in the CveDetail record, similar to the CPE table processing.
* Additional debugging support has been added
* The now unused CveResources table will be deleted in a later revision
[YOCTO #13121]
Signed-off-by: David Reyna <David.Reyna@windriver.com>
|
|
Change the new defect call to use named parameters. This will
enhance the readability and better allow for future changes.
Also, pass the CVE list and defect 'reason' so that the defect
integation tool can use that for the defect record and/or
use in creating its own version of the defect 'summary'.
Signed-off-by: David Reyna <David.Reyna@windriver.com>
|
|
Enable the feature of creating defects from investigations. Consolidate
into one defect creation method for both investigations and CVE
triage.
Enhance the "srtool_defect.py" sample tools to simulate creating
new defects.
Fix the sample "srtool_jira.py" tool new defect creation to support the
new "defect_tag" variable.
Signed-off-by: David Reyna <David.Reyna@windriver.com>
|
|
Update the 'urlpatterns' processing to use the master app.
Also, update the YP master app to include a url and view class, plus
provide a default YP landing page, and abtract the default logo display.
Signed-off-by: David Reyna <David.Reyna@windriver.com>
|
|
Transition the datasource scanning from 'datasource_org' to
the new master app environment variable, so that it all works
off of one key.
Also, add a sample logo for ACME, plus fix datasource trace details.
Signed-off-by: David Reyna <David.Reyna@windriver.com>
|
|
The SRTool allows users to substitute an alternate master application
instead of the default "yp" in order to customize their instance to
their organization.
This is done by:
(a) Creating a datasource directory under bin
(b) Defining a "datasource.json" file
(c) Defining 'export SRT_MAIN_APP="<app>"' in "srtool_env.sh"
This environment files are scanned by 'bin/srt', and if such an
alternate master app is found it pre-empts the default 'yp'.
This value is set via the environment because "lib/srtmain/settings.py"
is the file that sets the app (and this the URL) ordering, and it is
processed before any database is attached.
To disable the alternate main app, simply rename its "datasource.json"
file and it will be ignored for the next start.
The sample alternate app "acme" is provided to demonstrate this facility.
Additionally, a development tool 'bin/dev_tools/master_app.sh' has been
added to help switch between master apps, to aid testing.
$ ./stop.sh
$ ./master_app.sh acme
$ ./start.sh
... test ...
$ ./stop.sh
$ ./master_app.sh yp
$ ./start.sh
Other included fixes:
* Fix the ACME JSON files formating
* Remove ACME "_sample" from all but "datasource.json_sample"
* Fix tabs to spaces in "srt"
* Add global contect values to views::managedcontextprocessor so
that other app templates can share them
Signed-off-by: David Reyna <David.Reyna@windriver.com>
|
|
Signed-off-by: David Reyna <David.Reyna@windriver.com>
|
|
* Move the "Fetch Alt Sources" out of the authenticated user block
* Connect "Documentation" to the new User wiki page
* Minor typos and debugging line fixes
Signed-off-by: David Reyna <David.Reyna@windriver.com>
|
|
|
|
Fixes:
* Support Django development head in version check
(e.g. '2.2.dev20181217100344')
* Remove the single quotes around the comments content
* Include Documentation/Export links for Guest users
* Allow 'ip:port/acme' to link to 'acme_hello'
Signed-off-by: David Reyna <David.Reyna@windriver.com>
|
|
Signed-off-by: David Reyna <David.Reyna@windriver.com>
|
|
Run 'bin/common/srtool_sanity_test.py -i' to get a
quick sanity test of the database content and the
running SRTool server instance.
Development helper tools are provided in 'bin/dev_tools'
Signed-off-by: David Reyna <David.Reyna@windriver.com>
|
|
Ideally, these are all centralised.
|
|
|
|
|
|
|
|
|
|
|
|
Support Django-2.2:
Move 'django.core.urlresolvers' to 'django.urls'
Disable 'register.assignment_tag' tags
Move settings 'MIDDLEWARE_CLASSES' to 'MIDDLEWARE'
Move urlpatterns 'include' to 'path'
Move 'regex.pattern' to 'pattern.regex.pattern'
Maintain Django-1.11 support
General Fixes:
Fix commit for notify_categories
Add more error halt checks during lsupdates
Add explicit 'on_delete=models.CASCADE' for all ForeignKey's
Fix 'get_defect_tag' processing
[YOCTO #13091]
Signed-off-by: David Reyna <David.Reyna@windriver.com>
|
|
1. Create the sample "bin/acme_sample" organization data source,
to assist companies in adopting and customizing SRTool.
2. Add error detection and halting to the startup datasource scripts.
Signed-off-by: David Reyna <David.Reyna@windriver.com>
|
|
Signed-off-by: David Reyna <David.Reyna@windriver.com>
|
|
Changes:
Repartition the data sources
Reconfigure the data sources into self-contained directories under the "bin" directory.
Implement dynamic data source discovery and import
Remove all hard coded data source data (e.g. fixtures, data, CVE lookups)
Add license files to all data sources
Django User model
Add "users" Django application dir
Login page
Self create user account page
Password change page
User access and delete management
CVE
Name sorting by hidden 'name_sort' field (CVE-nnnn-0nnnnnn)
CVE Triage
Auto import reserved CVEs
Add MITRE CVE records where NIST missing
Add data source count to triage page
Easy checkbox toggle by clicking any field
Triage any CVE status category (not just new)
Assign to any CVE status category
Object create/delete
Create/Delete Vulnerablities
Create/Delete Investigations from Vulnerablity page
Add "Historical" CVE status
When bootstraping system, all CVEs older than 60 days preset to "Historical"
Add CVEs withint 60 days preset to "New"
Can be overridden by defect and systaining status imports
Preadd Debian data for "New" CVEs
Abstraction
Add generic Product mappings to defect system ("defect_tag": defect prefix)
Add generic Product mappings to product system ("product_tag": product reference, related)
Manage functions via "srt" script
For example add superuser
Normalize Vulnerability to Investigation mapping
Replace orm_vulnerabilityproduct with orm_vulnerabilitytoinvestigation
General
Enable the 'srtool-requirements.txt' Django test
Speed the CVE scoring by pre-fetching the datasources
Progress display cleanup
Move and update srtool_defect prototype to 'bin/yp'
Signed-off-by: David Reyna <David.Reyna@windriver.com>
|
|
Extend the NIST CPE scanning to also accept "cpe_match" as a table
for included CPEs (CVEs >= 2018).
[YOCTO #12996]
Signed-off-by: David Reyna <David.Reyna@windriver.com>
|
|
Fix the NIST update check to skip the UTC offset. Add the exec to
MITRE update to create data cache dir, update report.py for cve
data source schema changes.
[YOCTO #12996]
[YOCTO #12997]
Signed-off-by: David Reyna <David.Reyna@windriver.com>
|
|
Add the change list alert for non-vulnerable CVE triage. Fix
the mistaken error message from srtool_common.py.
Signed-off-by: David Reyna <David.Reyna@windriver.com>
|
|
Add the ability to edit the names for local
CVE's. This can specifically be used to create
placeholders for reserved CVEs (which are not
included in the NIST data) and then download the
data from the alternate CVE sources.
Signed-off-by: David Reyna <David.Reyna@windriver.com>
|
|
Summary:
* CVE management enhancements
* Lookup Mitre, Red Hat, Debian
* Local CVE's
* Edit CVE's
* CVE triage: create defects and notifications
* Packages
* Table, packages to C/V/I/D
* Filter model update
* Triage filtering script
* Notifications
* Upstream CVE changes
* Upstream defect changes
* Notices and reminders
* Creation
* Emails
* Productization
* Move DataSource to Init/Update/Lookup model
* Add defect details to Management home page
* Add author to notification records
* Move fixture data and SrtSettings to JSON files
* database column mappings for scripts generated directly from database
* srtool_utils.py: add [backup|restore]_cve_packages
Signed-off-by: David Reyna <David.Reyna@windriver.com>
|
|
* Improve the CVE detail caching
* Fix the "_FALLBACK" settings management
* Fix the YP "samples" fixture file
* Rename "srtool_cve.py" as "srtool_nist.cy"
Signed-off-by: David Reyna <David.Reyna@windriver.com>
|
|
* Add incremental NIST scans and import to CVE database
* Add modified NIST scans and import to CVE database
* Moved CVE details out of SRTool database to reduce size
* Add CVE details lookup in cached CVE upstream files
* Added edit support for Vulnerabilies and Investigations
* Comments
* Attachments and downloads
* Product list
* History audit trail
* Add Vulnerability and Investigation creation from CVE triage
* Add user id to session variables
* Add defect import placeholder script
* Modularize the fixture files for common versus site-specific setup
Signed-off-by: Moayer, Puya <Puya.Moayer@windriver.com>
Signed-off-by: David Reyna <David.Reyna@windriver.com>
|
|
executable
Signed-off-by: David Reyna <David.Reyna@windriver.com>
|
|
Signed-off-by: David Reyna <David.Reyna@windriver.com>
|
|
Signed-off-by: David Reyna <David.Reyna@windriver.com>
|
|
Fix typo in shutdown code to kill threads when "kill -0" is not enough.
Use the '--noreload' flag for 'runserver' so that there are no extra
and unaccounted threads.
[YOCTO #12555]
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add a 'nobuild' option for starting Toaster without the project
and hosted builds support. This allows a Toaster host to provide
local build statistics without opening the host to external users
building projects.
[YOCTO #12315]
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Toaster needs the ability to allow custom extensions to execute when
Toaster is started and stopped. Toaster will look for a custom
extension script in the fixtures directory and execute any applicable
hooks.
[YOCTO #11938]
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We were bridging the gap between the server and UI here by calling a
bb.siggen.find_siginfo, a function defined and set on that module from
the metadata. This worked from the UI side before but since the recent
server changes is no longer accessible. Create a new command so this can
execute on the server side and return the result by way of a new event.
(We're still running compare_sigfiles() on the signature generator but
that isn't quite the same thing and does still work.)
Fixes [YOCTO #11844].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|