Table of contents

Overview

TWIGS is an acronym for ThreatWorx Information Gathering Script. It is a python-based package that can be installed using “pip” i.e. python package manager. Twigs can help discover various classes of assets like cloud instances, servers, source code, containers and more. Also twigs can help run static analysis and other information security checks on your  source code as well. You can read more about twigs here. Note though most examples of twigs commands in this guide are depicted on Linux platform, twigs supports Windows platform as well. For discovering Windows assets, please refer here. Twigs supports python 2.7 onwards and python 3.x versions.

Installing twigs

Twigs is a python-based package and can be installed using ‘pip’ as below: 
sudo pip install twigs
Note:
  • It is recommended to use virtual environments to create isolated Python environments and reduce dependency conflicts. Please use the following commands to create new virtual environment depending on your python version and install twigs:
    # Python 3.x
    python3 -m venv twigs_env
    cd twigs_env
    source bin/activate
    pip3 install twigs

    # Python 2.7
    python -m virtualenv –python=/usr/bin/python2.7 twigs_env_2_7
    cd twigs_env_2_7
    source bin/activate
    pip install twigs

Initial setup of twigs

Twigs requires user details to connect to your ThreatWorx instance during the discovery process. You can either login into twigs or you can share user details via environment variables as an alternative to specify these on the command-line explicitly for every run. Both approaches are described here. twigs provides “login” and “logout” capability. This allows users to authenticate via twigs to their ThreatWorx instance. Once logged in, users do not need to provide their details while running twigs command every time. The login command is below: 
twigs login
Similarly to logout from twigs, you can run the command below: 
twigs logout
Alternatively you can perform the following steps one-time after installing twigs to simplify usage of twigs subsequently:
  • Configure your environment setup (via something like .bashrc for Bash Shell) to store typical parameters required with twigs.
  • These parameters are as follows:
    • TW_HANDLE —> User login to be used for asset ingestion. Note this user becomes the owner of the ingested assets.
    • TW_TOKEN —> API token for the specific user. You can find the value by following below steps:
      1. Login into ThreatWorx console
      2. Click on “Profile” in top menu
      3. Click on “Key Management” in left menu
      4. If you have not generated an API Key as yet, then click on “Generate New Key”, else click on “Copy to clipboard”. Note if you re-generate a new API key, then earlier key is disabled by default.
    • TW_INSTANCE —> Specify the specific TW instance provisioned for your organization.
  • Save these as follows in your profile script (like .bashrc):
export TW_HANDLE=<your login> export TW_TOKEN=<your API key> export TW_INSTANCE=<your TW instance>
  • Next time you login, these variables will be automatically set and will help reduce the clutter in your twigs command-line.
Note the order in which user details get used is as follows:
  • User details provided on the twigs command-line have highest preference.
  • User details of logged in user take next preference
  • User details from environment variables take last preference.

Current Platform/OS support for discovery

Twigs can discover various classes of assets as below:
  • Cloud assets from AWS, Azure and GCP
  • Docker containers
  • Import assets from a file (PDF or JSON)
  • Servers, desktops, laptops etc.
  • Source code repository
  • Ingest existing assets details from CMDB like ServiceNow, etc.
Twigs supports top platforms/OS as below:
  • RedHat
  • CentOS
  • Ubuntu
  • Debian
  • Amazon Linux
  • Windows
  • Mac OS

Cloud ready images

Twigs cloud native images are currently available for AWS, docker hub and Digital Ocean marketplaces. These images are free to download and use and are built and maintained by ThreatWorx to contain all the requirements and dependencies of twigs including any native CLI or other open source dependencies. Please check for updates in the section as we add such images for more cloud vendors and their marketplaces. For twigs docker it is necessary to have access to docker service inside the twigs instance. This can be easily accomplished by passing the docker unix socket to the instance. This can be accomplished by a a command like this while starting the twigs docker instance: 
docker run -it -v /var/run/docker.sock:/var/run/docker.sock threatworx/twigs:latest
The docker image already contains the other dependencies required for docker image discovery.

Common options

Twigs provides a bunch of common options i.e. irrespective of what type of assets are being discovered. These common options are optional as seen below: 
twigs [-h] [-v] [--handle HANDLE] [--token TOKEN] [--instance INSTANCE] [--run_id RUN_ID] 

             [--location LOCATION] [--tag_critical] [--tag TAG] [--no_auto_tags] 

             [--apply_policy APPLY_POLICY][--sbom SBOM] [--no_scan]
             [--email_report] [--schedule SCHEDULE] [--quiet]
Here is a quick description of these options:
  • run_id – Specify an unique identifier for this twigs run. Note – this identifier will help you group multiple runs for a specific type of discovery together and these will be shown together in the I3 Portal. If you don’t specify a “run_id”, then discovery “mode” will be used by default.
  • location – Specify location for discovered asset(s)
  • tag_critical – Mark the asset(s) as business critical
  • tag – You can use this option multiple times on the command line to add multiple tags to the asset(s)
  • no_auto_tags – Disable auto tagging of assets with standard classification tags. Only user specified tags will be applied.
  • apply_policy – You can use this option to specify name of a policy when you use twigs in your CI/CD pipeline to make policy based decisions like fail the build if any “DoNow” priority vulnerability impacts are discovered or any strong copyleft violations are found. For multiple policies, specify a comma-separated list of policy names.
  • sbom – Use this option to specify path to a SBOM file to save the asset(s)
  • no_scan – Indicate that you don’t wish to start a vulnerability assessment for the discovered asset(s)
  • email_report – Once the vulnerability assessment is done, then you will automatically be emailed a copy of the vulnerability assessment report.
  • schedule – Run this twigs command at specified schedule (in crontab format)
  • quiet – Do not display informational messages on the console during twigs run

Cloud Discovery

Twigs supports cloud native, agent less discovery of all 3 major cloud providers – AWS, Azure and GCP.

AWS

Overview Twigs supports cloud-native discovery for AWS i.e. twigs can ingest asset inventory gathered by AWS Systems Manager. Pre-requisites One needs to configure AWS Systems Manager to report asset inventory which is subsequently ingested by twigs. For more details on how to configure AWS Systems Manager, please refer to the links below: Steps involved After you have configured AWS Systems Manager to gather inventory, then you can run twigs to ingest this collected inventory into your ThreatWorx instance by following the below mentioned steps:
  • Open a new shell / terminal
  • Check that twigs is installed and running properly by running below command:
 twigs aws -h
  • Keep following AWS details handy to run the command:
    • AWS Account Identifier (AWS_ACCOUNT)
    • AWS Access Key (AWS_ACCESS_KEY)
    • AWS Secret Key (AWS_SECRET_KEY)
    • AWS Region (AWS_REGION)
    • AWS S3 Bucket (AWS_S3_BUCKET)
  • Run the command below:
      twigs aws --aws_account AWS_ACCOUNT 

                --aws_access_key AWS_ACCESS_KEY 

                --aws_secret_key AWS_SECRET_KEY 

                --aws_region AWS_REGION 

                --aws_s3_bucket AWS_S3_BUCKET 

                [--enable_tracking_tags]
  • It is suggested that you enable_tracking_tags, which allows you to easily identify AWS cloud instances in ThreatWorx
  • Note AWS cloud discovery may require some time depending on the number of EC2 instances in your AWS cloud setup.
  • After discovery is complete, you can login into ThreatWorx Console to view the newly discovered assets.
Help video: https://youtu.be/pYzHU7izRdU

ECR

Overview Twigs supports discovery of container images from AWS Container Registry (ECR). Pre-requisites AWS CLI is required, please install it by following the steps mentioned here for your Operating System. Note you need to configure using  AWS CLI and login into your ECR using “docker login” as well. You can inventory all images in your ECR by specifying registry name or single image by specifying fully qualified image name (with optional tag). Steps involved You can run twigs to ingest this collected inventory into your ThreatWorx instance by following the below mentioned steps:
  • Open a new shell / terminal
  • Check that twigs is installed and running properly by running below command:
twigs ecr -h
  • Configure AWS CLI for the first time.
  • Perform “docker login” as described here.
  • You can run the command mentioned below:
twigs ecr [--registry REGISTRY] [--image IMAGE] [--repository_type {public,private}] [--tmp_dir TMP_DIR] [--check_vuln CHECK_VULN] [--check_all_vulns]
  • For information on vulnerabilities supported by twigs plugins, refer here.
  • After discovery is complete, you can login into ThreatWorx Console to view the newly discovered assets.

Azure

Overview Twigs supports cloud-native discovery for Azure i.e. twigs can ingest asset inventory gathered by Azure in your Log Analytics Workspace. Pre-requisites Setting up a Azure Monitor in your Azure subscription requires some steps, you can refer to the documentation below: Steps involved After you have configured Azure Monitor to collect Azure VMs data in a Log Analytics Workspace, you can run twigs to ingest this collected inventory into your ThreatWorx instance by following the below mentioned steps:
  • Open a new shell / terminal
  • Check that twigs is installed and running properly by running below command:
 twigs azure -h
  • You need the following information to run twigs command:
    • Azure Tenant Identifier (AZURE_TENANT_ID)
    • Azure Application Identifier (AZURE_APPLICATION_ID)
    • Azure Application Key (AZURE_APPLICATION_KEY)
    • Azure Subscription (AZURE_SUBSCRIPTION)
    • Azure Resource Group (AZURE_RESOURCE_GROUP)
    • Azure Log Analytics Workspace (AZURE_WORKSPACE)
  • You can get these details from Azure Portal.
  • If you do not know values for (AZURE_SUBSCRIPTION, AZURE_RESOURCE_GROUP, AZURE_WORKSPACE), then simply run twigs with no values for those and twigs will list out possible values (as shown below) by querying your Azure subscription. You can then select the right value.
twigs azure --azure_tenant_id “MY_TENANT_ID” --azure_application_id “MY_APPLICATION_ID” --azure_application_key “MY_APPLICATION_KEY”

INFO     Started new run...

INFO     Using handle specified in "TW_HANDLE" environment variable...
INFO     Using token specified in "TW_TOKEN" environment variable...

INFO     Using instance specified in "TW_INSTANCE" environment variable...

INFO     Getting access token...
Missing details for subscription/resource group/workspace....

Available subscriptions with resource group and workspace details as below:

Subscription: MY_SUBSCRIPTION

 ** Resource group: MY_RESOURCE_GROUP1

 ** Resource group: MY_RESOURCE_GROUP2

 ** Resource group: MY_RESOURCE_GROUP3

 ** Workspace: MY_LOG_ANALYTICS_WORKSPACE
Please re-run twigs with appropriate values for subscription, resource group and workspace.
  • Run the command as shown below:
twigs azure --azure_tenant_id AZURE_TENANT_ID

                   --azure_application_id AZURE_APPLICATION_ID

                   --azure_application_key AZURE_APPLICATION_KEY

                   --azure_subscription AZURE_SUBSCRIPTION

                   --azure_resource_group AZURE_RESOURCE_GROUP

                   --azure_workspace AZURE_WORKSPACE
                   [—enable_tracking_tags]
  • It is suggested that you enable_tracking_tags, which allows you to easily identify Azure cloud instances in ThreatWorx.
  • Note Azure cloud discovery may require some time depending on the number of VM instances in your Azure cloud setup.
  • After discovery is complete, you can login into ThreatWorx Console to view the newly discovered assets.
Help video: https://youtu.be/DyMrxYscADw

ACR

Overview Twigs supports discovery of container images from Azure Container Registry (ACR). Pre-requisites Azure CLI is required, please install it by following the steps mentioned here for your Operating System. Note you need to login using az CLI and login into your ACR using “docker login” as well. You can inventory all images in your ACR by specifying registry name or single image by specifying fully qualified image name (with tag). Steps involved You can run twigs to ingest this collected inventory into your ThreatWorx instance by following the below mentioned steps:
  • Open a new shell / terminal
  • Check that twigs is installed and running properly by running below command:
twigs acr -h
  • Sign in into Azure account using az CLI.
  • Perform “docker login” as described here.
  • You can run the command mentioned below:
twigs acr [--registry REGISTRY] [--image IMAGE] [--tmp_dir TMP_DIR] [--check_vuln CHECK_VULN] [--check_all_vulns]
  • For information on vulnerabilities supported by twigs plugins, refer here.
  • After discovery is complete, you can login into ThreatWorx Console to view the newly discovered assets.

GCP

Overview Twigs supports cloud-native discovery for Google Cloud Platform (GCP) i.e. using OS inventory management. Pre-requisites VM Manager needs to be enabled for the relevant GCP projects. VM Manager is a suite of tools provided by GCP that can be used to manage operating systems for large virtual machine (VM) fleets running Windows and Linux on Compute Engine. One of the features of the VM Manager is OS Inventory Management which enables GCP to keep an inventory of all VM instances and their associated metadata which can be used by ThreatWorx for vulnerability assessment. The easiest way to enable VM Manager for your project is to follow the instructions for any running VM on your GCP project. Enable VM Manager automatically for your project or VM Google Cloud SDK is required. Please install it on the system that will run twigs, by following instructions mentioned here for your Operating System. The SDK provides tools (like the ‘gcloud’ CLI) which are used by twigs for discovering VM instances. Steps involved Once VM Manager is enabled for your compute instances in GCP, you can run twigs to ingest this collected inventory into your ThreatWorx instance by following the below mentioned steps:
  • Open a new shell / terminal
  • Check that twigs is installed and running properly by running below command:
twigs gcp -h
  • Sign in into your Google Cloud Platform instance using gcloud CLI as described here on the box where you will be running twigs.
  • You can run the command mentioned below:
twigs gcp [--enable_tracking_tags]
  • It is suggested that you enable_tracking_tags, which allows you to easily identify projects associated with discovered compute instances
  • Note GCP cloud discovery may require some time depending on the number of compute instances in your GCP cloud setup.
  • After discovery is complete, you can login into ThreatWorx Console to view the newly discovered assets.
Help video: https://youtu.be/tGgKZcGFfZ4

GCR

Overview Twigs supports discovery of container images from Google Container Registry (GCR). Pre-requisites Google Cloud SDK is required, please install it by following instructions mentioned here for your Operating System. The SDK provides tools (like gcloud) which are used. You can inventory all images in your GCR repository by specifying repository URL or single image by specifying fully qualified image name (with tag / digest). Steps involved You can run twigs to ingest this collected inventory into your ThreatWorx instance by following the below mentioned steps:
  • Open a new shell / terminal
  • Check that twigs is installed and running properly by running below command:
twigs gcr -h
  • Sign in into your Google Cloud Platform instance using gcloud CLI as described here on the box where you will be running twigs.
  • You can run the command mentioned below:
twigs gcr [--repository REPOSITORY] [--image IMAGE] [--tmp_dir TMP_DIR] [--check_vuln CHECK_VULN] [--check_all_vulns]
  • For information on vulnerabilities supported by twigs plugins, refer here.
  • After discovery is complete, you can login into ThreatWorx Console to view the newly discovered assets.

Serverless functions discovery

Overview Twigs supports discovery of Serverless functions for Azure Functions and Google Cloud Functions. Support for AWS Lambda functions is coming soon. Each Serverless function is treated like an individual source code repository asset and can be assessed for issues like open source dependency vulnerabilities, static analysis (SAST), secrets and embedded passwords, license compliance and software composition analysis (SCA).  Refer to source code discovery section for more details. Azure Functions Twigs supports discovery of Azure Functions code for security vulnerabilities through your Azure subscription. Each Azure Function will be represented as an asset in the ThreatWorx console. Pre-requisites Azure CLI is required, please install it by following the steps mentioned here for your Operating System. An Azure user or service principal with access to the Azure Functions API is required. The best builtin role required for the service principal to access the Azure Functions is Website Contributor For static code analysis (SAST) tests, twigs uses an open source tool called semgrep. For more information on semgrep and how to download it, refer to this link. For security checks in your infrastructure-as-code (IaC) code, twigs uses an open source tool called checkov. For more information on checkov and how to download it, refer to this link. Steps involved
  • Open a new shell / terminal.
  • Check that twigs is installed and running properly by running below command:
twigs azure_functions -h
  • You can run the command as below:
twigs azure_functions
               [--secrets_scan] 

               [--enable_entropy]

               [--regex_rules_file REGEX_RULES_FILE]

               [--check_common_passwords]

               [--common_passwords_file COMMON_PASSWORDS_FILE]

               [--include_patterns INCLUDE_PATTERNS]

               [--include_patterns_file INCLUDE_PATTERNS_FILE]

               [--exclude_patterns EXCLUDE_PATTERNS]

               [--exclude_patterns_file EXCLUDE_PATTERNS_FILE]

               [--mask_secret]
               [--no_code]

               [--sast]

               [--iac_checks]
  • After discovery is complete, you can login into ThreatWorx Console to view the newly discovery Azure Function assets from your subscription.
  • Secrets such as tokens, keys etc. embedded in the Azure Function source code can be found using –secrets_scan option. A default set of regex rules are used for this which can be substituted for your regex rules file. A more advanced (but noisy) way is to use –enable_entropy to detect secrets based on text patterns.
  • The –mask_secret option will hide any secret strings that are collected from the Azure Function source code.
  • The –no_code option will prevent any function code snippet from being collected or transmitted as part of the discovery. The source file and line numbers will be collected and transmitted.
  • Common passwords embedded in source code can be scanned for using the –check_common_passwords option. You may provide your own list of common passwords as a simple text file using –common_passwords_file option.
  • The include_ and exclude_ options can be used to specify or skip certain file name patterns or directories while scanning the Azure Function deployment.
Google Cloud Functions Twigs supports discovery of Google Cloud Functions code for security vulnerabilities through your GCP subscription. Each Google Cloud Function will be represented as an asset in the ThreatWorx console. Pre-requisites Google Cloud SDK is required, please install it by following instructions mentioned here for your Operating System. The SDK provides tools (like gcloud, gsutil etc.) which are used to discover the Google Functions. Following GCP IAM permissions are required for the gsutil command:
ls Projects storage.buckets.list storage.buckets.get5 storage.buckets.getIamPolicy6
ls Buckets, Objects storage.objects.list storage.objects.get5 storage.objects.getIamPolicy10
cp Objects storage.objects.list2 (for the destination bucket) storage.objects.get (for the source objects) storage.objects.create (for the destination bucket) storage.objects.delete3 (for the destination bucket)
For static code analysis (SAST) tests, twigs uses an open source tool called semgrep. For more information on semgrep and how to download it, refer to this link. For security checks in your infrastructure-as-code (IaC) code, twigs uses an open source tool called checkov. For more information on checkov and how to download it, refer to this link. Steps involved
  • Open a new shell / terminal.
  • Check that twigs is installed and running properly by running below command:
twigs gcloud_functions -h
  • You can run the command as below:
twigs gcloud_functions
               [--secrets_scan] 

               [--enable_entropy]

               [--regex_rules_file REGEX_RULES_FILE]

               [--check_common_passwords]

               [--common_passwords_file COMMON_PASSWORDS_FILE]

               [--include_patterns INCLUDE_PATTERNS]

               [--include_patterns_file INCLUDE_PATTERNS_FILE]

               [--exclude_patterns EXCLUDE_PATTERNS]

               [--exclude_patterns_file EXCLUDE_PATTERNS_FILE]

               [--mask_secret]
               [--no_code]

               [--sast]

               [--iac_checks]
  • After discovery is complete, you can login into ThreatWorx Console to view the newly discovered Google Cloud Function assets from your subscription.
  • Secrets such as tokens, keys etc. embedded in the Google Cloud Function source code can be found using –secrets_scan option. A default set of regex rules are used for this which can be substituted for your regex rules file. A more advanced (but noisy) way is to use –enable_entropy to detect secrets based on text patterns.
  • The –mask_secret option will hide any secret strings that are collected from the Google Cloud Function source code.
  • The –no_code option will prevent any function code snippet from being collected or transmitted as part of the discovery. The source file and line numbers will be collected and transmitted.
  • Common passwords embedded in source code can be scanned for using the –check_common_passwords option. You may provide your own list of common passwords as a simple text file using –common_passwords_file option.
  • The include_ and exclude_ options can be used to specify or skip certain file name patterns or directories while scanning the Google Cloud Function deployment.

Host discovery

Overview Twigs can discover hosts in two ways as below:
  • Discover the current host where is twigs is running. This is called as local host discovery and covered in more detail here.
  • Discover multiple hosts remotely. This is called as remote host discovery and covered in more detail here.
During host discovery, you can perform SSH audit and Host benchmarks as well. Local Host Discovery Overview Host discovery (local) is a fairly straightforward process. It needs twigs to be installed on the required host. Pre-requisites Twigs should be installed on the required host. Steps involved Once you have twigs installed on the required host, then you can follow the steps below for discovery local host as an asset in ThreatWorx:
  • Open a new shell / terminal
  • Check that twigs is installed and running properly by running below command:
 twigs host -h
  • You can run the command as below:
twigs host [--assetid ASSETID] [--assetname ASSETNAME] [--no_ssh_audit] [--no_host_benchmark] [--check_vuln CHECK_VULN] [--check_all_vulns]
  • For information on vulnerabilities supported by twigs plugins, refer here.
  • After discovery is complete, you can login into ThreatWorx Console to view the newly discovery asset.
Help video: https://youtu.be/OKJ-DxXwanA Remote hosts discovery Overview Twigs can help discover multiple hosts easily using remote hosts discovery. Pre-requisites Twigs remote discovery for hosts uses a CSV (comma-separate values) file which provides details about the hosts to be discovered. The CSV format has support for specifying individual remote hosts via hostname or IP address and you can specify a CIDR (Classless Inter-Domain Routing) or subnet range to discover hosts in your GCP cloud. You can read more details about the format of the CSV file here. It is recommended that you secure the credentials shared in the CSV file using the ‘—secure’ option provided by twigs. This can done by following the steps below:
  • Assume that you have created remote_hosts.csv which contains credentials in clear text.
  • Run the following command to secure the file:
twigs host --host_list remote_hosts.csv --secure
  • Open the remote_hosts.csv file to confirm that the credentials are secured if you want.
Steps involved You can follow the steps below for remote hosts discovery:
  • Open a new shell / terminal
  • Check that twigs is installed and running properly by running below command:
 twigs host -h
  • You need the following information to run twigs command:
    • remote_hosts.csv file created earlier as mentioned in pre-requisites section
  • Run the twigs command as below:
twigs host --remote_hosts_csv <<PATH_TO_REMOTE_HOSTS_CSV>> [--password PASSWORD] [--no_ssh_audit] [--no_host_benchmark] [--check_vuln CHECK_VULN] [--check_all_vulns]
  • For information on vulnerabilities supported by twigs plugins, refer here.
  • The discovery process may take some time depending on the number of hosts to be discovered.
  • After discovery is complete, you can login into ThreatWorx Console to view the newly discovered assets.

Container discovery

Overview Twigs supports discovering docker container images as an asset in ThreatWorx. This will discover source code assets from your docker container as well. For more details on source code assets, refer to Source Code Asset discovery in twigs. Pre-requisites Docker CLI and service are required for discovering docker images. For more details on installing docker CLI and service for your operating system, refer to this link. Steps involved You can follow the steps below to discover your docker container images/instances as assets in ThreatWorx:
  • Open a new shell / terminal
  • Check that twigs is installed and running properly by running below command:
 twigs docker -h
  • You can run the command below:
twigs docker --image IMAGE [--assetid ASSETID] [--assetname ASSETNAME] [--tmp_dir TMP_DIR] [--start_instance] [--check_vuln CHECK_VULN] [--check_all_vulns]
where IMAGE has format (repo:tag) and if tag is not specified, then “latest” is assumed.
  • For information on vulnerabilities supported by twigs plugins, refer here.
  • After discovery is complete, you can login into ThreatWorx console to view the newly discovered asset.

Kubernetes discovery

Overview Twigs supports discovering your Kubernetes environment. This essentially discovers docker container images in your Kubernetes environment as asset(s) in ThreatWorx. This will discover source code assets from your docker container as well. For more details on source code assets, refer to Source Code Asset discovery in twigs. Pre-requisites Docker CLI and service are required for discovering docker images. For more details on installing docker CLI and service for your operating system, refer to this link. For discovering your helm charts, you need to have helm installed. Steps involved You can follow the steps below to discover your Kubernetes environment as assets in ThreatWorx:
  • Open a new shell / terminal
  • Check that twigs is installed and running properly by running below command:
 twigs k8s -h
  • You can run the command below:
twigs k8s [-h] (--deployment_yaml DEPLOYMENT_YAML | --helm_chart HELM_CHART) [--tmp_dir TMP_DIR] [--check_vuln CHECK_VULN] [--check_all_vulns]
where DEPLOYMENT_YAML is path to Kubernetes Deployment Manifest Definition YAML file or you can specify HELM_CHART (as path to local helm chart folder or as repo/chartname).
  • For information on vulnerabilities supported by twigs plugins, refer here.
  • After discovery is complete, you can login into ThreatWorx console to view the newly discovered asset(s).

VMware discovery

Overview twigs allows you to discover and model VMware products and services including vCenter and ESXi as assets. This discovery is done using VMware vSphere python SDK. Only vCenter and ESXi instances will be discovered currently. Support will be enhanced to discover other VMware products and licenses over time. Other information such as cluster, datacenter will also be discovered and used for automatic tagging via twigs. Pre-requisites VMware vSphere python SDK – pyvmomi is required. This will be installed automatically as part of twigs installation or can be separately installed via the ‘pip’ command. Read-only access to a vSphere / vCenter console is required as user name and password. Steps involved Use the VMware discovery mode in twigs as follows: 
twigs vmware --host <vCenter/vSphere HOSTNAME/IP> --user <vCenter/vSphere USER> [--password <password>]
Password can also be set in an environment variable as ‘VCENTER_PASSWD’. If available, the environment variable will have precedence over the command line password.

SBOM-based discovery

Overview SBOM stands for Software Bill Of Materials. SBOM-based discovery mode in twigs allows you to ingest assets specified in SBOM artifact to ThreatWorx. Currently supported SBOM standards and formats are as follows:
  • CycloneDX – JSON
  • SPDX, SPDX Lite – tagvalue
  • ThreatWorx (proprietary) – JSON, CSV
Pre-requisites You need to have SBOM artifact. Steps involved The steps involved to discover assets from SBOM artifact are as below:
  • Open a new shell / terminal.
  • Check that twigs is installed and running properly by running below command:
twigs sbom -h
  • You can run the command as below:
twigs sbom [-h] --input INPUT [--standard {cyclonedx,spdx,threatworx}] [--format {json,tagvalue,csv}] [--assetid ASSETID] [--assetname ASSETNAME]
  • where INPUT is the path to the SBOM document
  • After discovery is complete, you can login into ThreatWorx Console to view the newly discovery assets.

nmap based discovery

Overview Twigs supports discovering assets from your environment using nmap. Pre-requisites You need to have nmap installed on your host (where you will be running twigs). Steps involved The steps involved to discover assets using nmap in your environment are as below:
  • Open a new shell / terminal.
  • Check that twigs is installed and running properly by running below command:
twigs nmap -h
  • You can run the command below:
twigs nmap [-h] [--hosts HOSTS] [--timing {0,1,2,3,4,5}] [--discovery_scan_type {N,S,A,U,Y,O,E,P,M}] [--discovery_port_list DISCOVERY_PORT_LIST]
where HOSTS can be hostname, IP address or CIDR For details on DISCOVERY_SCAN_TYPE, DISCOVERY_PORT_LIST values, refer to link.
  • After discovery is complete, you can login into ThreatWorx Console to view the newly discovery assets.

Discover source code as an asset

Overview Twigs can discover your source code as an asset. For a quick primer, read this. You can discover a single git repo or local repo folder or all repositories for a given GitHub User. Note for the latter i.e. discover all repositories for GitHub User; you need to have GitHub CLI “gh” installed. Supported package / dependency managers for source code discovery are:
  • Python [pip]
  • Java Script [npm, yarn]
  • Ruby [ruby]
  • Java [maven, gradle, jar]
  • .NET/C# [nuget]
  • DLL (* for vulnerability assessment only)
  • Rust [cargo]
There are multiple functionalities provided as below:
  • Identify vulnerabilities – This helps you identify any vulnerabilities in 3rd party libraries / packages used in your source code project. You can indicate to twigs whether you are interested in tracking vulnerabilities in direct (shallow level) or indirect (deep level) dependencies.
  • License compliance – You need to know how licensing of the open source components (libraries/packages) used in your software project impact you. For example – one cannot release a commercial software product built using open source components with a restrictive license
  • Code secrets – Twigs can be used to identify any secrets that are inadvertently embedded in your source code. Twigs can identify secrets using any or all of the three approach mentioned below:
    • Entropy – Detect secrets by automatically identifying high entropy strings in your source code.
    • Regular Expressions – twigs provides support for regular expressions for identifying standard secrets (like OAuth tokens, JWT tokens, etc.). You can specify your own custom regular expressions in a file if needed.
    • Common Passwords – twigs provides support for identifying common passwords from a top 500 common passwords list. You can provide your own common passwords file, if needed.
  • Static Application Security Testing (SAST) – You can perform SAST checks on your source code projects.
  • Infrastructure as Code (IaC) security tests on cloudformation, terraform, kubernetes, serverless, arm, terraform_plan, helm templates.
Pre-requisites For static code analysis (SAST) tests, twigs uses an open source tool called semgrep. For more information on semgrep and how to download it, refer to this link. For security checks in your infrastructure-as-code (IaC) code, twigs uses an open source tool called checkov. For more information on checkov and how to download it, refer to this link. Steps involved The steps involved to discover your source code as an asset are as below:
  • Open a new shell / terminal.
  • Check that twigs is installed and running properly by running below command:
twigs repo -h
  • You can run the command as below:
twigs repo  (--repo REPO | --gh_user GH_USER)

               [--branch BRANCH]

               [--type {pip,ruby,yarn,nuget,npm,maven,gradle,dll,jar,cargo}]

               [--level {shallow,deep}] 

               [--include_unused_dependencies]

               [--assetid ASSETID]

               [--assetname ASSETNAME] 

               [--secrets_scan] 

               [--enable_entropy]

               [--regex_rules_file REGEX_RULES_FILE]

               [--check_common_passwords]

               [--common_passwords_file COMMON_PASSWORDS_FILE]

               [--include_patterns INCLUDE_PATTERNS]

               [--include_patterns_file INCLUDE_PATTERNS_FILE]

               [--exclude_patterns EXCLUDE_PATTERNS]

               [--exclude_patterns_file EXCLUDE_PATTERNS_FILE]

               [--mask_secret]
               [--no_code]

               [--sast]

               [--iac_checks]
  • After discovery is complete, you can login into ThreatWorx Console to view the newly discovery asset from your source code.
  • Use the branch option to specify an optional branch of remote git repo (applicable for single git repo discovery only).
  • For certain dependency managers like npm, you can control the dependency levels to inspect by using the –level [shallow|deep] option.
  • twigs can determine the type of dependency manager by looking at the source code. But in case you want twigs to discover dependencies of a certain type, you can use the –type option.
  • For certain technologies (like npmjs), only used direct dependencies are considered by default (i.e. dependencies referenced in the source code). Use to the –include_unused_dependencies switch to consider all dependencies, note this may introduce false positives.
  • Secrets such as tokens, keys etc. embedded in the source code can be found using –secrets_scan option. A default set of regex rules are used for this which can be substituted for your regex rules file. A more advanced (but noisy) way is to use –enable_entropy to detect secrets based on text patterns.
  • The –mask_secret option will hide any secret strings that are collected from the source code.
  • The –no_code option will prevent any code snippet from being collected or transmitted as part of the discovery. The source file and line numbers will be collected and transmitted.
  • Common passwords embedded in source code can be scanned for using the –check_common_passwords option. You may provide your own list of common passwords as a simple text file using –common_passwords_file option.
  • The include_ and exclude_ options can be used to specify or skip certain file name patterns or directories while scanning the source code repository.

Discover GitHub Enterprise repositories as assets

Overview Twigs can discover your source code as an asset. For a quick primer, read this. You can discover all repositories for a GitHub Enterprise Organization. Note you need to have GitHub CLI “gh” installed. Supported package / dependency managers for source code discovery are:
  • Python [pip]
  • Java Script [npm, yarn]
  • Ruby [ruby]
  • Java [maven, gradle, jar]
  • .NET/C# [nuget]
  • DLL (* for vulnerability assessment only)
  • Rust [cargo]
There are multiple functionalities provided as below:
  • Identify vulnerabilities – This helps you identify any vulnerabilities in 3rd party libraries / packages used in your source code project. You can indicate to twigs whether you are interested in tracking vulnerabilities in direct (shallow level) or indirect (deep level) dependencies.
  • License compliance – You need to know how licensing of the open source components (libraries/packages) used in your software project impact you. For example – one cannot release a commercial software product built using open source components with a restrictive license
  • Code secrets – Twigs can be used to identify any secrets that are inadvertently embedded in your source code. Twigs can identify secrets using any or all of the three approach mentioned below:
    • Entropy – Detect secrets by automatically identifying high entropy strings in your source code.
    • Regular Expressions – twigs provides support for regular expressions for identifying standard secrets (like OAuth tokens, JWT tokens, etc.). You can specify your own custom regular expressions in a file if needed.
    • Common Passwords – twigs provides support for identifying common passwords from a top 500 common passwords list. You can provide your own common passwords file, if needed.
  • Static Application Security Testing (SAST) – You can perform SAST checks on your source code projects.
  • Infrastructure as Code (IaC) security tests on cloudformation, terraform, kubernetes, serverless, arm, terraform_plan, helm templates.
Pre-requisites For static code analysis (SAST) tests, twigs uses an open source tool called semgrep. For more information on semgrep and how to download it, refer to this link. For security checks in your infrastructure-as-code (IaC) code, twigs uses an open source tool called checkov. For more information on checkov and how to download it, refer to this link. Steps involved The steps involved to discover your source code as an asset are as below:
  • Open a new shell / terminal.
  • Check that twigs is installed and running properly by running below command:
twigs ghe -h
  • You can run the command as below:
twigs ghe  --gh_org GH_ORG

               [--type {pip,ruby,yarn,nuget,npm,maven,gradle,dll,jar,cargo}]

               [--level {shallow,deep}] 

               [--include_unused_dependencies]

               [--assetid ASSETID]

               [--assetname ASSETNAME] 

               [--secrets_scan] 

               [--enable_entropy]

               [--regex_rules_file REGEX_RULES_FILE]

               [--check_common_passwords]

               [--common_passwords_file COMMON_PASSWORDS_FILE]

               [--include_patterns INCLUDE_PATTERNS]

               [--include_patterns_file INCLUDE_PATTERNS_FILE]

               [--exclude_patterns EXCLUDE_PATTERNS]

               [--exclude_patterns_file EXCLUDE_PATTERNS_FILE]

               [--mask_secret]
               [--no_code]

               [--sast]

               [--iac_checks]
  • After discovery is complete, you can login into ThreatWorx Console to view the newly discovery asset from your source code.
  • For certain dependency managers like npm, you can control the dependency levels to inspect by using the –level [shallow|deep] option.
  • twigs can determine the type of dependency manager by looking at the source code. But in case you want twigs to discover dependencies of a certain type, you can use the –type option.
  • For certain technologies (like npmjs), only used direct dependencies are considered by default (i.e. dependencies referenced in the source code). Use to the –include_unused_dependencies switch to consider all dependencies, note this may introduce false positives.
  • Secrets such as tokens, keys etc. embedded in the source code can be found using –secrets_scan option. A default set of regex rules are used for this which can be substituted for your regex rules file. A more advanced (but noisy) way is to use –enable_entropy to detect secrets based on text patterns.
  • The –mask_secret option will hide any secret strings that are collected from the source code.
  • The –no_code option will prevent any code snippet from being collected or transmitted as part of the discovery. The source file and line numbers will be collected and transmitted.
  • Common passwords embedded in source code can be scanned for using the –check_common_passwords option. You may provide your own list of common passwords as a simple text file using –common_passwords_file option.
  • The include_ and exclude_ options can be used to specify or skip certain file name patterns or directories while scanning the source code repository.

Discover assets from ServiceNow

Overview You can point twigs to your ServiceNow (CMDB) to ingest existing asset information to ThreatWorx. Pre-requisites None. Steps involved You can follow the steps below to ingest your existing asset inventory from ServiceNow to ThreatWorx:
  • Open a new shell / terminal.
  • Check that twigs is installed and running properly by running below command:
twigs servicenow -h
  • Keep following ServiceNow instance details handy to run the command:
    • ServiceNow User (SNOW_USER)
    • ServiceNow User Password (SNOW_USER_PWD)
    • ServiceNow Instance (SNOW_INSTANCE)
  • You can run the command below:
     twigs servicenow  --snow_user SNOW_USER 

                         --snow_user_pwd SNOW_USER_PWD

                         --snow_instance SNOW_INSTANCE

                        [--enable_tracking_tags]
  • It is suggested that you enable_tracking_tags, which allows you to easily identify assets ingested from ServiceNow in ThreatWorx
  • After discovery is complete, you can login into ThreatWorx Console to view the newly discovery asset from your source code.

DAST checks on web applications as assets

Overview Using this feature you can run DAST checks on your web applications and track them in ThreatWorx. This allows you to track your web applications as assets in ThreatWorx Pre-requisites A supported DAST tool like arachni, skipfish, arachni etc. is required. This needs to be installed on the host where twigs is running. Steps involved You can follow the steps below to discover your web application as an asset and run DAST checks on it
  • Open a new shell / terminal.
  • Check that twigs is installed and running properly by running below command:
twigs dast -h
  • Make sure you have a supported DAST plugin installed. Currently supported tools are:
  • Download and install either of the plugin CLIs and note the install location (folder)
  • Note your web application url and make sure it is accessible from the host where twigs is running.
  • Then run the command below:
twigs dast  --url WEB_APPLICATION_URL

                         --assetid UNIQUE_ASSET_ID

[--plugin arachni | skipfish ]

[--pluginpath <path to plugin CLI install location> ]

                        [--args EXTRA_ARGS_FOR_PLUGIN_CLI ]

                        [--assetname NAME_LABEL_FOR_ASSET ]
  • After discovery is complete, you can login into ThreatWorx Console to view the newly discovery web application asset as well as results of the DAST tests.
  • These tests could be automated as part of a CI/CD pipeline to get your web application regularly tested.
  • Twigs will automatically mark/resolve any fixed issues that were discovered as part of a previous run.

Run and track docker-bench results

Overview Using this feature you can run docker bench CIS checks on your docker hosts and containers and track them in ThreatWorx. This allows you to track your docker host as a regular linux host as well as running docker-bench tests on it. Pre-requisites docker-bench is required. This can be downloaded from https://github.com/docker/docker-bench-security. Steps involved
  • Open a new shell / terminal.
  • Check that twigs is installed and running properly by running below command:
twigs docker_cis -h
  • Make sure you have docker-bench downloaded
  • Note the location of docker-bench-security.sh which is available in the downloaded copy of docker-bench-security
  • You can run the command below:
twigs docker_cis  [--assetid UNIQUE_ASSET_ID]
                  [--assetname NAME_LABEL_FOR_ASSET ]
                  [--docker_bench_home DOCKER_BENCH_HOME]
  • Asset id and name are optional. Docker_bench_home defaults to /usr/share/docker-bench-security
  • After discovery is complete, you can login into ThreatWorx Console to view the newly discovery host as well as results of the docker bench tests.
  • Twigs will automatically mark/resolve any fixed issues that were discovered as part of a previous run.

Run and track CIS benchmarks for AWS

Overview Using this feature you can run CIS benchmark tests (v1.2.0) for your AWS cloud. This includes CIS level 1 and level 2 checks for AWS as specified here: https://www.cisecurity.org/benchmark/amazon_web_services/ Pre-requisites Prowler is an open source tool that allows you to run CIS benchmarks for AWS. This tool is free and can be downloaded from https://github.com/toniblyx/prowler. Please install any dependencies required by prowler tool. Steps involved
  • Open a new shell / terminal.
  • Check that twigs is installed and running properly by running below command:
twigs aws_cis -h
  • Make sure you have Prowler downloaded
  • Note the location of prowler executable which is available in the downloaded copy of prowler. You can set environment variable “PROWLER_HOME” which points to Prowler download directory, instead of specifying “–prowler_home” argument on command-line.
  • You can run the command below:
twigs aws_cis  --aws_access_key AWS_ACCESS_KEY

                      --aws_secret_key AWS_SECRET_KEY

                      --assetid UNIQUE_ASSET_ID

                      --assetname NAME_LABEL_FOR_ASSET

                      --prowler_home HOME_DIRECTORY_FOR_PROWLER
  • Asset id is not optional. Use a unique identifier for your AWS cloud instance as an asset.
  • After discovery is complete, you can login into ThreatWorx Console to view the newly discovered AWS instance as an asset as well as results of the CIS benchmark tests.
  • Twigs will automatically mark/resolve any fixed issues that were discovered as part of a previous run.

Run and track audit for AWS

Overview Using this feature you can run comprehensive audit for your AWS cloud. This covers checks for multiple regulatory standards like HIPAA, PCI, GDPR, etc. Pre-requisites Prowler is an open source tool that allows you to run audit for AWS. This tool is free and can be downloaded from https://github.com/toniblyx/prowler. Please install any dependencies required by prowler tool. Steps involved
  • Open a new shell / terminal.
  • Check that twigs is installed and running properly by running below command:
twigs aws_audit -h
  • Make sure you have Prowler downloaded
  • Note the location of prowler executable which is available in the downloaded copy of prowler. You can set environment variable “PROWLER_HOME” which points to Prowler download directory, instead of specifying “–prowler_home” argument on command-line.
  • You can run the command below:
twigs aws_audit  --aws_access_key AWS_ACCESS_KEY

                      --aws_secret_key AWS_SECRET_KEY

                      --assetid UNIQUE_ASSET_ID

                      --assetname NAME_LABEL_FOR_ASSET

                      --prowler_home HOME_DIRECTORY_FOR_PROWLER
  • Asset id is not optional. Use a unique identifier for your AWS cloud instance as an asset.
  • After discovery is complete, you can login into ThreatWorx Console to view the newly discovered AWS instance as an asset as well as results of the AWS Audit.
  • Twigs will automatically mark/resolve any fixed issues that were discovered as part of a previous run.

Run and track CIS benchmarks for Azure

Overview Using this feature you can run CIS benchmark tests (v1.0.0) for your Azure cloud. This includes CIS level 1 and level 2 checks for Azure as specified here: https://www.cisecurity.org/benchmark/azure/ Pre-requisites Azure CLI is required, please install it by following the steps mentioned here for your Operating System. Steps involved
  • Open a new shell / terminal.
  • Check that twigs is installed and running properly by running below command:
twigs azure_cis -h
  • Sign in into your Azure instance using Azure CLI as described here on the host where you will be running twigs.
  • You can run the command below:
twigs azure_cis --assetid UNIQUE_ASSET_ID

                       --assetname NAME_LABEL_FOR_ASSET
  • Asset id is not optional. Use a unique identifier for your Azure cloud instance as an asset.
  • After discovery is complete, you can login into ThreatWorx Console to view the newly discovered Azure instance as an asset as well as results of the CIS benchmark tests.
  • Twigs will automatically mark/resolve any fixed issues that were discovered as part of a previous run.

Run and track CIS benchmarks for Google Cloud Platform

Overview Using this feature you can run CIS benchmark tests (v1.1.0) for your Google Cloud Platform. This includes CIS level 1 and level 2 checks for Google Cloud Platform as specified here: https://www.cisecurity.org/benchmark/google_cloud_computing_platform/. Pre-requisites Google Cloud SDK is required, please install it by following instructions mentioned here for your Operating System. The SDK provides tools (like gcloud, gsutil, bq, etc.) which are used. The following GCP APIs should be enabled for all projects that need to be checked:
  • cloudkms.googleapis.com
  • cloudresourcemanager.googleapis.com
  • compute.googleapis.com
  • dns.googleapis.com
  • iam.googleapis.com
  • logging.googleapis.com
  • monitoring.googleapis.com
  • sqladmin.googleapis.com
  • storage-api.googleapis.com
The following permissions are required to run the CIS benchmark: On organization level:
  • resourcemanager.organizations.get
  • resourcemanager.projects.get
  • resourcemanager.projects.getIamPolicy
  • resourcemanager.folders.get
On project level:
  • cloudkms.cryptoKeys.get
  • cloudkms.cryptoKeys.getIamPolicy
  • cloudkms.cryptoKeys.list
  • cloudkms.keyRings.list
  • cloudsql.instances.get
  • cloudsql.instances.list
  • compute.firewalls.get
  • compute.firewalls.list
  • compute.instances.get
  • compute.instances.list
  • compute.networks.get
  • compute.networks.list
  • compute.projects.get
  • compute.regions.list
  • compute.sslPolicies.get
  • compute.sslPolicies.list
  • compute.subnetworks.get
  • compute.subnetworks.list
  • compute.targetHttpsProxies.get
  • compute.targetHttpsProxies.list
  • compute.zones.list
  • dns.managedZones.get
  • dns.managedZones.list
  • iam.serviceAccountKeys.list
  • iam.serviceAccounts.list
  • logging.logMetrics.list
  • logging.sinks.get
  • logging.sinks.list
  • monitoring.alertPolicies.list
  • resourcemanager.projects.get
  • resourcemanager.projects.getIamPolicy
  • storage.buckets.get
  • storage.buckets.getIamPolicy
  • storage.buckets.list
Steps involved
  • Open a new shell / terminal.
  • Check that twigs is installed and running properly by running below command:
twigs gcp_cis -h
  • Sign in into your Google Cloud Platform instance using gcloud CLI as described here on the box where you will be running twigs.
  • You can run the command below:
twigs gcp_cis --assetid UNIQUE_ASSET_ID
                     [--assetname NAME_LABEL_FOR_ASSET]
                     [--projects PROJECT_IDS]
                     [--expanded]
                     [--custom_ratings CUSTOM_RATINGS]
  • Asset id is not optional. Use a unique identifier for your Google Cloud Platform instance as an asset.
  • Use –projects to specify an optional comma separated list of GCP project IDs to run the tests against.
  • By default there is a single issue reported for each CIS benchmark test. Use –expanded switch to create an issue for each violation (rather than each CIS benchmark test).
  • To override default ratings / severity for reported CIS issues, you can specify custom ratings / severity level for specific CIS benchmark tests in a custom JSON file. This JSON file will have rating as the key and a list of associated GCP CIS test identifiers in a dictionary. A sample format of the JSON file is provided here. This sample file also helps enumerate all test identifiers along with their default rating / severity.
  • After discovery is complete, you can login into ThreatWorx Console to view the newly discovered Google Cloud Platform instance as an asset as well as results of the CIS benchmark tests.
  • Twigs will automatically mark/resolve any fixed issues that were discovered as part of a previous run.

Run and track CIS benchmarks for Oracle Cloud Infrastructure (OCI)

Overview Using this feature you can run CIS benchmark tests for your Oracle Cloud Infrastructure (OCI). This includes CIS level 1 and level 2 checks for OCI as specified here. Pre-requisites Please setup OCI SDK and CLI as described here. Twigs uses default configuration file from following location “~/.oci/config” and it refers to the DEFAULT profile in the configuration file.
Steps involved
  • Open a new shell / terminal.
  • You can run the command: twigs oci_ci --assetid UNIQUE_ASSET_ID --assetname NAME_LABEL_FOR_ASSET
  • Asset id is not optional. Use a unique identifier for your OCI cloud instance as an asset.
  • After discovery is complete, you can login into ThreatWorx Console to view the newly discovered OCI instance as an asset as well as results of the CIS benchmark tests.
  • If you do not wish to run Oracle Best Practice checks, then you can include the “–no_obp” switch as follows: twigs oci_ci --assetid UNIQUE_ASSET_ID --assetname NAME_LABEL_FOR_ASSET --no_obp
  • Twigs will automatically mark/resolve any fixed issues that were discovered as part of a previous run.

Run and track CIS benchmarks for Kubernetes

Overview Using this feature you can run CIS benchmark tests for your Kubernetes cluster. This includes CIS level 1 and level 2 checks for Kubernetes as specified here. Pre-requisites None Steps involved
  • Open a new shell / terminal.
  • Check that twigs is installed and running properly by running below command:
twigs k8s_cis -h
  • You can run the command below:
twigs k8s_cis --assetid UNIQUE_ASSET_ID
                     [--assetname NAME_LABEL_FOR_ASSET]
                     --target {master, worker}
                     [--custom_ratings CUSTOM_RATINGS]
  • Asset id is not optional. Use a unique identifier for your Kubernetes cluster node i.e. master/worker as an asset.
  • Specify which tests to run based on the type of the Kubernetes node i.e. master or worker using the ‘target’ option.
  • To override default ratings / severity for reported CIS issues, you can specify custom ratings / severity level for specific CIS benchmark tests in a custom JSON file. This JSON file will have rating as the key and a list of associated Kubernetes test identifiers in a dictionary. A sample format of the JSON file is provided here.
  • After discovery is complete, you can login into ThreatWorx Console to view the newly discovered Kubernetes node as an asset as well as results of the CIS benchmark tests.
  • Twigs will automatically mark/resolve any fixed issues that were discovered as part of a previous run.

Run and track CIS benchmarks for GKE

Overview Using this feature you can run CIS benchmark tests for your GKE cluster. This includes CIS level 1 and level 2 checks for GKE as specified here. Currently these benchmarks are only available for GKE worker nodes since there is no way to access the GKE cluster master nodes. Pre-requisites None Steps involved
  • Open a new shell / terminal.
  • Check that twigs is installed and running properly by running below command:
twigs gke_cis -h
  • You can run the command below:
twigs gke_cis --assetid UNIQUE_ASSET_ID
                     [--assetname NAME_LABEL_FOR_ASSET]
                     --target {master, worker}
                     [--custom_ratings CUSTOM_RATINGS]
  • Asset id is not optional. Use a unique identifier for your GKE cluster node i.e. master/worker as an asset.
  • Specify which tests to run based on the type of the GKE node i.e. master or worker using the ‘target’ option. Only the ‘worker’ target is supported for now.
  • To override default ratings / severity for reported CIS issues, you can specify custom ratings / severity level for specific CIS benchmark tests in a custom JSON file. This JSON file will have rating as the key and a list of associated GKE CIS test identifiers in a dictionary. A sample format of the JSON file is provided here.
  • After discovery is complete, you can login into ThreatWorx Console to view the newly discovered Kubernetes node as an asset as well as results of the CIS benchmark tests.
  • Twigs will automatically mark/resolve any fixed issues that were discovered as part of a previous run.

Appendix

Windows asset discovery

Windows assets can be discovered using twigs Powershell script (twigs.ps1). Twigs Powershell script for windows discovery is digitally signed for security reasons. You can discover local or remote Windows hosts using Twigs Powershell script. Follow the steps below to perform discovery of Windows assets using twigs Powershell script:
  • Logon to the Windows machine for discovery
  • Download twigs PowerShell script from here
  • Install signing certificate for the Powershell script
    • Right click the Powershell script in Windows explorer and select Properties
    • In the Properties window, click on “Digital Signatures” tab
    • In the “Signature List” table, select row for “ThreatWorx”
    • Click on “Details” and then “View Certificate”
    • Click on “Install Certificate…” to install the certificate for “Local Machine” in “Trusted Root Certificate Authorities” and “Trusted Publishers” store.
  • Note you will need API token key to perform discovery.
  • Please ensure that ExecutionPolicy allows running Powershell scripts. To view current execution policy, please run ‘Get-ExecutionPolicy’. Refer to note below for details on how to change it.
  • The typical command parameters to twigs Powershell script are as below
.\twigs.ps1 [[-mode] <String>] [[-remote_hosts_csv] <String>] [[-host_list]
<String>] [[-password] <String>] [-handle] <String> [[-token] <String>] [[-instance] <String>] [[-out] <String>]
[[-assetid] <String>] [[-assetname] <String>] [[-tags] <String[]>] [-tag_critical] [-no_scan] [-email_report]
[<CommonParameters>]
  • To view above help run ‘.\twigs.ps1 -?’ or for detailed help message run ‘Get-Help .\twigs.ps1 -detailed’
  • For remote host discovery, you need to specify remote Windows host details in a CSV file. This CSV file has the format as below:
hostname,userlogin,userpwd
<Windows machine name or IP or CIDR>,<Login ID>,<Password>
MyWindowsMachine,MyWindowsDomain\MyWindowsUser,MyWindowsUserPassword
  • Note it is recommended that you secure the password in the CSV file by using ‘-host_list’ option provided by Twigs Powershell script.
  • Specify appropriate arguments and run twigs Powershell script.
Help video: https://youtu.be/BuTvbERm_Xs Note you may need to relax the execution policy for Powershell scripts on your Windows box. Typically you can set it using Set-ExecutionPolicy as below: 
Set-ExecutionPolicy RemoteSigned
or 
Set-ExecutionPolicy AllSigned

CSV Format for Asset Ingestion

twigs supports ingesting assets from CSV files. CSV files contain “comma-separated values” records consisting of multiple columns. The format of CSV file for asset ingestion has following columns in that order:
  • Asset Id – This is a required field and specifies the Identifier for the asset
  • Asset Name – This is a required field and specifies the Name for the asset
  • Asset Type – This is a required field and specifies the Type of the asset. Typical asset types are as below:
    • Windows
    • Red Hat
    • Ubuntu
    • CentOS
    • Debian
  • Owner – This is an optional field and can be empty. This field contains the handle of the user who will own the asset. If empty, handle of the user running twigs will be set as owner for the asset.
  • Tags – This is a required field and specifies the Tags associated with the asset. There can be multiple tags which are separated by semi-colon (“;”). There are some required tags as below:
    • OS_RELEASE – This is a required field. It contains the complete PRETTY_NAME for Linux flavors. For Windows it is OS name. For example:
      •  OS_RELEASE:Red Hat Enterprise Linux 8.1 (Ootpa)
      • OS_RELEASE:Microsoft Windows Server 2016 Datacenter
    • OS_VERSION – This is required for Windows assets and contains the OS version details (including Service Pack information). For example:
      •  OS_VERSION:6.1.7601 Service Pack 1 Build 7601
      • OS_VERSION:10.0.14393 N/A Build 14393
    • OS_RELEASE_ID – This is required for Windows assets, mainly newer versions of Windows which include a Release ID. For example:
      • OS_RELEASE_ID:1607
      • OS_RELEASE_ID:1809
    • OS_ARCH – This is required for Windows assets and specifies the machine architecture. For example:
      • OS_ARCH:64-bit x64-based PC
  • Products – This is a required field and specifies the list of products for the asset. There can be multiple products in the list which are separated by semi-colon (“;”)
  • Patches – This is a required field for Windows assets. Each patch is signaled as “KB” number such as “KB43215”. Multiple patches can be specified separated by semi-colon (“;”)
The CSV file starts with CSV header record as show below: Asset Id,Asset Name,Asset Type,Owner,Tags,Products,Patches

Troubleshooting discovery issues

Here are some common troubleshooting tips for twigs:
  • Twigs generates a log file called twigs.log in the current working directory. You can look at this file for any error or warning messages.
  • Twigs can export all discovered assets to specified JSON file using “–sbom” switch. You can view/edit this file if needed.
  • If the host running twigs has no connectivity to the internet, then twigs will be unable to automatically push the discovered assets to your ThreatWorx instance. Restore internet connectivity and then run twigs again. Note you can also import the generated JSON file if needed.
If you observe any issues with twigs, please write to us at support@threatworx.io and we would be happy to help.

Twigs command-line usage

Please refer to twigs documentation for command-line usage

Twigs source code

Twigs is open source. If you are interested in going through the source code of twigs, you can find it here.Orac