top of page

Forum Posts

threatwatch io
Dec 12, 2022
In FAQ and Integrations
Here are some common troubleshooting tips for twigs: Twigs generates a log file called twigs.log in the current working directory. You can look at this file for any error or warning messages. Twigs can export all discovered assets to specified JSON file using “–out” switch. You can view/edit this file if needed. If the host running twigs has no connectivity to the internet, then twigs will be unable to automatically push the discovered assets to your ThreatWorx instance. Restore internet connectivity and then run twigs again. Note you can also import the generated JSON file if needed. If you observe any issues with twigs, please write to us at support@threatwatch.io and we would be happy to help.
0
0
16
threatwatch io
Dec 12, 2022
In Corporate Assets Discovery
Windows assets can be discovered using twigs Powershell script (twigs.ps1). Twigs Powershell script for windows discovery is digitally signed for security reasons. You can discover local or remote Windows hosts using Twigs Powershell script. Steps Follow the steps below to perform discovery of Windows assets using twigs Powershell script: Logon to the Windows machine for discovery Download twigs PowerShell script from here Install signing certificate for the Powershell script Right click the Powershell script in Windows explorer and select Properties In the Properties window, click on “Digital Signatures” tab In the “Signature List” table, select row for “ThreatWorx” Click on “Details” and then “View Certificate” Click on “Install Certificate…” to install the certificate for “Local Machine” in “Trusted Root Certificate Authorities” and “Trusted Publishers” store. Note you will need API token key to perform discovery. Please ensure that ExecutionPolicy allows running Powershell scripts. To view current execution policy, please run ‘Get-ExecutionPolicy’. Refer to note below for details on how to change it. The typical command parameters to twigs Powershell script are as below .\twigs.ps1 [[-mode] <String>] [[-remote_hosts_csv] <String>] [[-host_list] <String>] [[-password] <String>] [-handle] <String> [[-token] <String>] [[-instance] <String>] [[-out] <String>] [[-assetid] <String>] [[-assetname] <String>] [[-tags] <String[]>] [-tag_critical] [-no_scan] [-email_report] [<CommonParameters>] To view above help run ‘.\twigs.ps1 -?’ or for detailed help message run ‘Get-Help .\twigs.ps1 -detailed’ For remote host discovery, you need to specify remote Windows host details in a CSV file. This CSV file has the format as below: hostname,userlogin,userpwd <Windows machine name or IP or CIDR>,<Login ID>,<Password> MyWindowsMachine,MyWindowsDomain\MyWindowsUser,MyWindowsUserPassword Note it is recommended that you secure the password in the CSV file by using ‘-host_list’ option provided by Twigs Powershell script. Specify appropriate arguments and run twigs Powershell script. Note you may need to relax the execution policy for Powershell scripts on your Windows box. Typically you can set it using Set-ExecutionPolicy as below: Set-ExecutionPolicy RemoteSigned or Set-ExecutionPolicy AllSigned
0
0
17
threatwatch io
Dec 12, 2022
In Source Code Discovery
Overview Twigs can discover your source code as an asset. For a quick primer, read this. You can discover all repositories for a GitHub Enterprise Organization. Note you need to have GitHub CLI “gh” installed. Supported package / dependency managers for source code discovery are: Python [pip] Java Script [npm, yarn] Ruby [ruby] Java [maven, gradle, jar] .NET/C# [nuget] DLL (* for vulnerability assessment only) Rust [cargo] There are multiple functionalities provided as below: Identify vulnerabilities – This helps you identify any vulnerabilities in 3rd party libraries / packages used in your source code project. You can indicate to twigs whether you are interested in tracking vulnerabilities in direct (shallow level) or indirect (deep level) dependencies. License compliance – You need to know how licensing of the open source components (libraries/packages) used in your software project impact you. For example – one cannot release a commercial software product built using open source components with a restrictive license Code secrets – Twigs can be used to identify any secrets that are inadvertently embedded in your source code. Twigs can identify secrets using any or all of the three approach mentioned below: Entropy – Detect secrets by automatically identifying high entropy strings in your source code. Regular Expressions – twigs provides support for regular expressions for identifying standard secrets (like OAuth tokens, JWT tokens, etc.). You can specify your own custom regular expressions in a file if needed. Common Passwords – twigs provides support for identifying common passwords from a top 500 common passwords list. You can provide your own common passwords file, if needed. Static Application Security Testing (SAST) – You can perform SAST checks on your source code projects. Infrastructure as Code (IaC) security tests on cloudformation, terraform, kubernetes, serverless, arm, terraform_plan, helm templates. Pre-requisites For static code analysis (SAST) tests, twigs uses an open source tool called semgrep. For more information on semgrep and how to download it, refer to this link. For security checks in your infrastructure-as-code (IaC) code, twigs uses an open source tool called checkov. For more information on checkov and how to download it, refer to this link. Steps involved The steps involved to discover your source code as an asset are as below: Open a new shell / terminal. Check that twigs is installed and running properly by running below command: twigs ghe -h You can run the command as below: twigs ghe --gh_org GH_ORG [--type {pip,ruby,yarn,nuget,npm,maven,gradle,dll,jar,cargo}] [--level {shallow,deep}] [--include_unused_dependencies] [--assetid ASSETID] [--assetname ASSETNAME] [--secrets_scan] [--enable_entropy] [--regex_rules_file REGEX_RULES_FILE] [--check_common_passwords] [--common_passwords_file COMMON_PASSWORDS_FILE] [--include_patterns INCLUDE_PATTERNS] [--include_patterns_file INCLUDE_PATTERNS_FILE] [--exclude_patterns EXCLUDE_PATTERNS] [--exclude_patterns_file EXCLUDE_PATTERNS_FILE] [--mask_secret] [--no_code][--sast][--iac_checks] After discovery is complete, you can login into ThreatWorx Console to view the newly discovery asset from your source code. Pro Tips: For certain dependency managers like npm, you can control the dependency levels to inspect by using the –level [shallow|deep] option. twigs can determine the type of dependency manager by looking at the source code. But in case you want twigs to discover dependencies of a certain type, you can use the –type option. For certain technologies (like npmjs), only used direct dependencies are considered by default (i.e. dependencies referenced in the source code). Use to the –include_unused_dependencies switch to consider all dependencies, note this may introduce false positives. Secrets such as tokens, keys etc. embedded in the source code can be found using –secrets_scan option. A default set of regex rules are used for this which can be substituted for your regex rules file. A more advanced (but noisy) way is to use –enable_entropy to detect secrets based on text patterns. The –mask_secret option will hide any secret strings that are collected from the source code. The –no_code option will prevent any code snippet from being collected or transmitted as part of the discovery. The source file and line numbers will be collected and transmitted. Common passwords embedded in source code can be scanned for using the –check_common_passwords option. You may provide your own list of common passwords as a simple text file using –common_passwords_file option. The include_ and exclude_ options can be used to specify or skip certain file name patterns or directories while scanning the source code repository.
0
0
23
threatwatch io
Dec 12, 2022
In Source Code Discovery
Overview Twigs can discover your source code as an asset. For a quick primer, read this. You can discover a single git repo or local repo folder or all repositories for a given GitHub User. Note for the latter i.e. discover all repositories for GitHub User; you need to have GitHub CLI “gh” installed. Supported package / dependency managers for source code discovery are: Python [pip] Java Script [npm, yarn] Ruby [ruby] Java [maven, gradle, jar] .NET/C# [nuget] DLL (* for vulnerability assessment only) Rust [cargo] There are multiple functionalities provided as below: Identify vulnerabilities – This helps you identify any vulnerabilities in 3rd party libraries / packages used in your source code project. You can indicate to twigs whether you are interested in tracking vulnerabilities in direct (shallow level) or indirect (deep level) dependencies. License compliance – You need to know how licensing of the open source components (libraries/packages) used in your software project impact you. For example – one cannot release a commercial software product built using open source components with a restrictive license Code secrets – Twigs can be used to identify any secrets that are inadvertently embedded in your source code. Twigs can identify secrets using any or all of the three approach mentioned below: Entropy – Detect secrets by automatically identifying high entropy strings in your source code. Regular Expressions – twigs provides support for regular expressions for identifying standard secrets (like OAuth tokens, JWT tokens, etc.). You can specify your own custom regular expressions in a file if needed. Common Passwords – twigs provides support for identifying common passwords from a top 500 common passwords list. You can provide your own common passwords file, if needed. Static Application Security Testing (SAST) – You can perform SAST checks on your source code projects. Infrastructure as Code (IaC) security tests on cloudformation, terraform, kubernetes, serverless, arm, terraform_plan, helm templates. Pre-requisites For static code analysis (SAST) tests, twigs uses an open source tool called semgrep. For more information on semgrep and how to download it, refer to this link. For security checks in your infrastructure-as-code (IaC) code, twigs uses an open source tool called checkov. For more information on checkov and how to download it, refer to this link. Steps The steps involved to discover your source code as an asset are as below: Open a new shell / terminal. Check that twigs is installed and running properly by running below command: twigs repo -h You can run the command as below: twigs repo (--repo REPO | --gh_user GH_USER)[--branch BRANCH] [--type {pip,ruby,yarn,nuget,npm,maven,gradle,dll,jar,cargo}] [--level {shallow,deep}][--include_unused_dependencies] [--assetid ASSETID] [--assetname ASSETNAME][--secrets_scan] [--enable_entropy] [--regex_rules_file REGEX_RULES_FILE] [--check_common_passwords] [--common_passwords_file COMMON_PASSWORDS_FILE] [--include_patterns INCLUDE_PATTERNS] [--include_patterns_file INCLUDE_PATTERNS_FILE] [--exclude_patterns EXCLUDE_PATTERNS] [--exclude_patterns_file EXCLUDE_PATTERNS_FILE] [--mask_secret][--no_code][--sast][--iac_checks] After discovery is complete, you can login into ThreatWorx Console to view the newly discovery asset from your source code. Pro Tips: Use the branch option to specify an optional branch of remote git repo (applicable for single git repo discovery only). For certain dependency managers like npm, you can control the dependency levels to inspect by using the –level [shallow|deep] option. twigs can determine the type of dependency manager by looking at the source code. But in case you want twigs to discover dependencies of a certain type, you can use the –type option. For certain technologies (like npmjs), only used direct dependencies are considered by default (i.e. dependencies referenced in the source code). Use to the –include_unused_dependencies switch to consider all dependencies, note this may introduce false positives. Secrets such as tokens, keys etc. embedded in the source code can be found using –secrets_scan option. A default set of regex rules are used for this which can be substituted for your regex rules file. A more advanced (but noisy) way is to use –enable_entropy to detect secrets based on text patterns. The –mask_secret option will hide any secret strings that are collected from the source code. The –no_code option will prevent any code snippet from being collected or transmitted as part of the discovery. The source file and line numbers will be collected and transmitted. Common passwords embedded in source code can be scanned for using the –check_common_passwords option. You may provide your own list of common passwords as a simple text file using –common_passwords_file option. The include_ and exclude_ options can be used to specify or skip certain file name patterns or directories while scanning the source code repository.
0
0
32
threatwatch io
Dec 12, 2022
In Corporate Assets Discovery
Twigs can discover hosts in two ways as below: Local Discovery: Discover the current host where is twigs is running. This is called as local host discovery and covered in more detail here. Remote Discovery: Discover multiple hosts remotely. This is called as remote host discovery and covered in more detail here. During host discovery, you can perform SSH audit and Host benchmarks as well. Local Host Discovery Overview Host discovery (local) is a fairly straightforward process. It needs twigs to be installed on the required host. Pre-requisites Twigs should be installed on the required host. Steps Once you have twigs installed on the required host, then you can follow the steps below for discovery local host as an asset in ThreatWorx: Open a new shell / terminal Check that twigs is installed and running properly by running below command: twigs host -h You can run the command as below: twigs host [--assetid ASSETID] [--assetname ASSETNAME] [--no_ssh_audit] [--no_host_benchmark] [--check_vuln CHECK_VULN] [--check_all_vulns] For information on vulnerabilities supported by twigs plugins, refer here. After discovery is complete, you can login into ThreatWorx Console to view the newly discovery asset. Remote hosts discovery Overview Twigs can help discover multiple hosts easily using remote hosts discovery. Pre-requisites Twigs remote discovery for hosts uses a CSV (comma-separate values) file which provides details about the hosts to be discovered. The CSV format has support for specifying individual remote hosts via hostname or IP address and you can specify a CIDR (Classless Inter-Domain Routing) or subnet range to discover hosts in your GCP cloud. You can read more details about the format of the CSV file here. It is recommended that you secure the credentials shared in the CSV file using the ‘—secure’ option provided by twigs. This can done by following the steps below: Assume that you have created remote_hosts.csv which contains credentials in clear text. Run the following command to secure the file: twigs host --host_list remote_hosts.csv --secure Open the remote_hosts.csv file to confirm that the credentials are secured if you want. Steps You can follow the steps below for remote hosts discovery: Open a new shell / terminal Check that twigs is installed and running properly by running below command: twigs host -h You need the following information to run twigs command: remote_hosts.csv file created earlier as mentioned in pre-requisites section Run the twigs command as below: twigs host --remote_hosts_csv <<PATH_TO_REMOTE_HOSTS_CSV>> [--password PASSWORD] [--no_ssh_audit] [--no_host_benchmark] [--check_vuln CHECK_VULN] [--check_all_vulns] For information on vulnerabilities supported by twigs plugins, refer here. The discovery process may take some time depending on the number of hosts to be discovered. After discovery is complete, you can login into ThreatWorx Console to view the newly discovered assets.
0
0
27
threatwatch io
Dec 12, 2022
In Corporate Assets Discovery
Overview Twigs supports discovering assets from your environment using nmap. Pre-requisites You need to have nmap installed on your host (where you will be running twigs). Steps The steps involved to discover assets using nmap in your environment are as below: • Open a new shell / terminal. • Check that twigs is installed and running properly by running below command: twigs nmap -h • You can run the command below: twigs nmap [-h] [--hosts HOSTS] [--timing {0,1,2,3,4,5}] [--discovery_scan_type {N,S,A,U,Y,O,E,P,M}] [--discovery_port_list DISCOVERY_PORT_LIST] [--no_ssh_audit] where HOSTS can be hostname, IP address or CIDR TIMING allows fine grained control on performance. For details refer link.(https://nmap.org/book/man-performance.html) DISCOVERY_SCAN_TYPE allows user to customize the host discovery scan. For details refer link.(https://nmap.org/book/man-host-discovery.html) DISCOVERY_PORT_LIST can be used to specify port(s) to be used in the host discovery scan. • After discovery is complete, you can login into ThreatWorx Console to view the newly discovery assets.
0
0
15
threatwatch io
Dec 12, 2022
In SBOM Based Discovery
Overview SBOM stands for Software Bill Of Materials. SBOM-based discovery mode in twigs allows you to ingest assets specified in SBOM artifact to ThreatWorx. Currently supported SBOM standards and formats are as follows: CycloneDX – JSON SPDX, SPDX Lite – tagvalue ThreatWorx (proprietary) – JSON, CSV Pre-requisites You need to have SBOM artifact. Steps The steps involved to discover assets from SBOM artifact are as below: Open a new shell / terminal. Check that twigs is installed and running properly by running below command: twigs sbom -h You can run the command as below: twigs sbom [-h] --input INPUT [--standard {cyclonedx,spdx,threatworx}] [--format {json,tagvalue,csv}] [--assetid ASSETID] [--assetname ASSETNAME] where INPUT is the path to the SBOM document After discovery is complete, you can login into ThreatWorx Console to view the newly discovery assets.
0
0
60
threatwatch io
Dec 12, 2022
In Corporate Assets Discovery
Overview twigs allows you to discover and model VMware products and services including vCenter and ESXi as assets. This discovery is done using VMware vSphere python SDK. Only vCenter and ESXi instances will be discovered currently. Support will be enhanced to discover other VMware products and licenses over time. Other information such as cluster, datacenter will also be discovered and used for automatic tagging via twigs. Pre-requisites VMware vSphere python SDK – pyvmomi is required. This will be installed automatically as part of twigs installation or can be separately installed via the ‘pip’ command. Read-only access to a vSphere / vCenter console is required as user name and password. Steps Use the VMware discovery mode in twigs as follows: twigs vmware --host <vCenter/vSphere HOSTNAME/IP> --user <vCenter/vSphere USER> [--password <password>] Password can also be set in an environment variable as ‘VCENTER_PASSWD’. If available, the environment variable will have precedence over the command line password.
0
0
11
threatwatch io
Dec 12, 2022
In Corporate Assets Discovery
Overview twigs supports discovering your Kubernetes environment. This essentially discovers docker container images in your Kubernetes environment as asset(s) in ThreatWorx. This will discover source code assets from your docker container as well. For more details on source code assets, refer to Source Code Asset discovery in twigs. Pre-requisites Docker CLI and service are required for discovering docker images. For more details on installing docker CLI and service for your operating system, refer to this link. For discovering your helm charts, you need to have helm installed. Steps You can follow the steps below to discover your Kubernetes environment as assets in ThreatWorx: Open a new shell / terminal Check that twigs is installed and running properly by running below command: twigs k8s -h You can run the command below: twigs k8s [-h] (--deployment_yaml DEPLOYMENT_YAML | --helm_chart HELM_CHART) [--tmp_dir TMP_DIR] [--check_vuln CHECK_VULN] [--check_all_vulns] where DEPLOYMENT_YAML is path to Kubernetes Deployment Manifest Definition YAML file or you can specify HELM_CHART (as path to local helm chart folder or as repo/chartname). For information on vulnerabilities supported by twigs plugins, refer here. After discovery is complete, you can login into ThreatWorx console to view the newly discovered asset(s).
0
0
8
threatwatch io
Dec 12, 2022
In Container Discovery
Overview twigs supports discovering docker container images as an asset in ThreatWorx. This will discover source code assets from your docker container as well. For more details on source code assets, refer to Source Code Asset discovery in twigs. Pre-requisites Docker CLI and service are required for discovering docker images. For more details on installing docker CLI and service for your operating system, refer to this link. Steps You can follow the steps below to discover your docker container images/instances as assets in ThreatWorx: Open a new shell / terminal Check that twigs is installed and running properly by running below command: twigs docker -h You can run the command below: twigs docker --image IMAGE [--assetid ASSETID] [--assetname ASSETNAME] [--tmp_dir TMP_DIR] [--start_instance] [--check_vuln CHECK_VULN] [--check_all_vulns] where IMAGE has format (repo:tag) and if tag is not specified, then “latest” is assumed. For information on vulnerabilities supported by twigs plugins, refer here. After discovery is complete, you can login into ThreatWorx console to view the newly discovered asset.
0
0
16
threatwatch io
Dec 12, 2022
In Cloud Discovery
Twigs supports discovery of Serverless functions for Azure Functions and Google Cloud Functions. Support for AWS Lambda functions is coming soon. Each Serverless function is treated like an individual source code repository asset and can be assessed for issues like open source dependency vulnerabilities, static analysis (SAST), secrets and embedded passwords, license compliance and software composition analysis (SCA). Refer to source code discovery section for more details. Azure Functions Google Cloud Functions Azure Functions Overview twigs supports discovery of Azure Functions code for security vulnerabilities through your Azure subscription. Each Azure Function will be represented as an asset in the ThreatWorx console. Pre-requisites Azure CLI is required, please install it by following the steps mentioned here for your Operating System. An Azure user or service principal with access to the Azure Functions API is required. The best builtin role required for the service principal to access the Azure Functions is Website Contributor For static code analysis (SAST) tests, twigs uses an open source tool called semgrep. For more information on semgrep and how to download it, refer to this link. For security checks in your infrastructure-as-code (IaC) code, twigs uses an open source tool called checkov. For more information on checkov and how to download it, refer to this link. Steps Open a new shell / terminal. Check that twigs is installed and running properly by running below command: twigs azure_functions -h You can run the command as below: twigs azure_function [--secrets_scan][--enable_entropy] [--regex_rules_file REGEX_RULES_FILE][--check_common_passwords] [--common_passwords_file COMMON_PASSWORDS_FILE] [--include_patterns INCLUDE_PATTERNS] [--include_patterns_file INCLUDE_PATTERNS_FILE] [--exclude_patterns EXCLUDE_PATTERNS] [--exclude_patterns_file EXCLUDE_PATTERNS_FILE] [--mask_secret] [--no_code][--sast][--iac_checks] After discovery is complete, you can login into ThreatWorx Console to view the newly discovery Azure Function assets from your subscription. Secrets such as tokens, keys etc. embedded in the Azure Function source code can be found using –secrets_scan option. A default set of regex rules are used for this which can be substituted for your regex rules file. A more advanced (but noisy) way is to use –enable_entropy to detect secrets based on text patterns. The –mask_secret option will hide any secret strings that are collected from the Azure Function source code. The –no_code option will prevent any function code snippet from being collected or transmitted as part of the discovery. The source file and line numbers will be collected and transmitted. Common passwords embedded in source code can be scanned for using the –check_common_passwords option. You may provide your own list of common passwords as a simple text file using –common_passwords_file option. The include_ and exclude_ options can be used to specify or skip certain file name patterns or directories while scanning the Azure Function deployment. Google Cloud Functions Overview Twigs supports discovery of Google Cloud Functions code for security vulnerabilities through your GCP subscription. Each Google Cloud Function will be represented as an asset in the ThreatWorx console. Pre-requisites Google Cloud SDK is required, please install it by following instructions mentioned here for your Operating System. The SDK provides tools (like gcloud, gsutil etc.) which are used to discover the Google Functions. Following GCP IAM permissions are required for the gsutil command: ls Projects storage.buckets.list storage.buckets.get storage.buckets.get.IamPolicy ls Buckets, Objects storage.objects.list storage.objects.get storage.buckets.get.IamPolicy cp Objects storage.objects.list (for the destination bucket) storage.objects.get (for the source objects) storage.objects.create (for the destination bucket) storage.objects.delete (for the destination bucket) storage.objects.delete (for the destination bucket) For static code analysis (SAST) tests, twigs uses an open source tool called semgrep. For more information on semgrep and how to download it, refer to this link. For security checks in your infrastructure-as-code (IaC) code, twigs uses an open source tool called checkov. For more information on checkov and how to download it, refer to this link. Steps Open a new shell / terminal. Check that twigs is installed and running properly by running below command: twigs gcloud_functions -h You can run the command as below: twigs gcloud_functions [--secrets_scan][--enable_entropy] [--regex_rules_file REGEX_RULES_FILE][--check_common_passwords] [--common_passwords_file COMMON_PASSWORDS_FILE] [--include_patterns INCLUDE_PATTERNS] [--include_patterns_file INCLUDE_PATTERNS_FILE] [--exclude_patterns EXCLUDE_PATTERNS] [--exclude_patterns_file EXCLUDE_PATTERNS_FILE] [--mask_secret][--no_code][--sast][--iac_checks] After discovery is complete, you can login into ThreatWorx Console to view the newly discovered Google Cloud Function assets from your subscription. Secrets such as tokens, keys etc. embedded in the Google Cloud Function source code can be found using –secrets_scan option. A default set of regex rules are used for this which can be substituted for your regex rules file. A more advanced (but noisy) way is to use –enable_entropy to detect secrets based on text patterns. The –mask_secret option will hide any secret strings that are collected from the Google Cloud Function source code. The –no_code option will prevent any function code snippet from being collected or transmitted as part of the discovery. The source file and line numbers will be collected and transmitted. Common passwords embedded in source code can be scanned for using the –check_common_passwords option. You may provide your own list of common passwords as a simple text file using –common_passwords_file option. The include_ and exclude_ options can be used to specify or skip certain file name patterns or directories while scanning the Google Cloud Function deployment.
0
0
13
threatwatch io
Dec 12, 2022
In Container Discovery
twigs supports discovery of your container images available as part of any your cloud container registries. Amazon Elastic Container Registry (ECR) Azure Container Registry (ACR) Google Container Registry (GCR) Amazon Elastic Container Registry (ECR) Overview Twigs supports discovery of container images from AWS Container Registry (ECR). Pre-requisites AWS CLI is required, please install it by following the steps mentioned here for your Operating System. Note you need to configure using AWS CLI and login into your ECR using “docker login” as well. You can inventory all images in your ECR by specifying registry name or single image by specifying fully qualified image name (with optional tag). Steps You can run twigs to ingest this collected inventory into your ThreatWorx instance by following the below mentioned steps: Open a new shell / terminal Check that twigs is installed and running properly by running below command: twigs ecr -h Configure AWS CLI for the first time. Perform “docker login” as described here. You can run the command mentioned below: twigs ecr [--registry REGISTRY] [--image IMAGE] [--repository_type {public,private}] [--tmp_dir TMP_DIR] [--check_vuln CHECK_VULN] [--check_all_vulns] For information on vulnerabilities supported by twigs plugins, refer here. After discovery is complete, you can login into ThreatWorx Console to view the newly discovered assets. Azure Container Registry (ACR) Overview Twigs supports discovery of container images from Azure Container Registry (ACR). Pre-requisites Azure CLI is required, please install it by following the steps mentioned here for your Operating System. Note you need to login using az CLI and login into your ACR using “docker login” as well. You can inventory all images in your ACR by specifying registry name or single image by specifying fully qualified image name (with tag). Steps You can run twigs to ingest this collected inventory into your ThreatWorx instance by following the below mentioned steps: Open a new shell / terminal Check that twigs is installed and running properly by running below command: twigs acr -h Sign in into Azure account using az CLI. Perform “docker login” as described here. You can run the command mentioned below: twigs acr [--registry REGISTRY] [--image IMAGE] [--tmp_dir TMP_DIR] [--check_vuln CHECK_VULN] [--check_all_vulns] For information on vulnerabilities supported by twigs plugins, refer here. After discovery is complete, you can login into ThreatWorx Console to view the newly discovered assets. Google Container Registry (GCR) Overview Twigs supports discovery of container images from Google Container Registry (GCR). Pre-requisites Google Cloud SDK is required, please install it by following instructions mentioned here for your Operating System. The SDK provides tools (like gcloud) which are used. You can inventory all images in your GCR repository by specifying repository URL or single image by specifying fully qualified image name (with tag / digest). Steps You can run twigs to ingest this collected inventory into your ThreatWorx instance by following the below mentioned steps: Open a new shell / terminal Check that twigs is installed and running properly by running below command: twigs gcr -h Sign in into your Google Cloud Platform instance using gcloud CLI as described here on the box where you will be running twigs. You can run the command mentioned below: twigs gcr [--repository REPOSITORY] [--image IMAGE] [--tmp_dir TMP_DIR] [--check_vuln CHECK_VULN] [--check_all_vulns] For information on vulnerabilities supported by twigs plugins, refer here. After discovery is complete, you can login into ThreatWorx Console to view the newly discovered assets.
0
0
13
threatwatch io
Dec 12, 2022
In Cloud Discovery
twigs supports cloud native, agent less discovery of all 3 major cloud providers – AWS, Azure and GCP Lets look at each one of them in subsequent sections. AWS Cloud Discovery Azure Cloud Discovery GCP Cloud Discovery AWS Cloud Discovery Overview Twigs supports cloud-native discovery for AWS i.e. twigs can ingest asset inventory gathered by AWS Systems Manager. Pre-requisites One needs to configure AWS Systems Manager to report asset inventory which is subsequently ingested by twigs. For more details on how to configure AWS Systems Manager, please refer to the links below: AWS documentation on Systems Manager AWS documentation on how to inventory all managed instances in your AWS account ThreatWorx documentation on how to configure AWS Systems Manager Steps After you have configured AWS Systems Manager to gather inventory, then you can run twigs to ingest this collected inventory into your ThreatWorx instance by following the below mentioned steps: Open a new shell / terminal Check that twigs is installed and running properly by running below command: twigs aws -h Keep following AWS details handy to run the command: AWS Account Identifier (AWS_ACCOUNT) AWS Access Key (AWS_ACCESS_KEY) AWS Secret Key (AWS_SECRET_KEY) AWS Region (AWS_REGION) AWS S3 Bucket (AWS_S3_BUCKET) Run the command below: twigs aws --aws_account AWS_ACCOUNT --aws_access_key AWS_ACCESS_KEY --aws_secret_key AWS_SECRET_KEY --aws_region AWS_REGION --aws_s3_bucket AWS_S3_BUCKET [--enable_tracking_tags] It is suggested that you enable_tracking_tags, which allows you to easily identify AWS cloud instances in ThreatWorx Note AWS cloud discovery may require some time depending on the number of EC2 instances in your AWS cloud setup. After discovery is complete, you can login into ThreatWorx Console to view the newly discovered assets. Azure Cloud Discovery Overview Twigs supports cloud-native discovery for Azure i.e. twigs can ingest asset inventory gathered by Azure in your Log Analytics Workspace. Pre-requisites Setting up a Azure Monitor in your Azure subscription requires some steps, you can refer to the documentation below: Azure documentation for collecting data from an Azure VM with Azure Monitor ThreatWorx Azure Integration Steps After you have configured Azure Monitor to collect Azure VMs data in a Log Analytics Workspace, you can run twigs to ingest this collected inventory into your ThreatWorx instance by following the below mentioned steps: Open a new shell / terminal Check that twigs is installed and running properly by running below command: twigs azure -h You need the following information to run twigs command: Azure Tenant Identifier (AZURE_TENANT_ID) Azure Application Identifier (AZURE_APPLICATION_ID) Azure Application Key (AZURE_APPLICATION_KEY) Azure Subscription (AZURE_SUBSCRIPTION) Azure Resource Group (AZURE_RESOURCE_GROUP) Azure Log Analytics Workspace (AZURE_WORKSPACE) You can get these details from Azure Portal. If you do not know values for (AZURE_SUBSCRIPTION, AZURE_RESOURCE_GROUP, AZURE_WORKSPACE), then simply run twigs with no values for those and twigs will list out possible values (as shown below) by querying your Azure subscription. You can then select the right value. twigs azure --azure_tenant_id “MY_TENANT_ID” --azure_application_id “MY_APPLICATION_ID” --azure_application_key “MY_APPLICATION_KEY”INFO Started new run...INFO Using handle specified in "TW_HANDLE" environment variable... INFO Using token specified in "TW_TOKEN" environment variable... INFO Using instance specified in "TW_INSTANCE" environment variable... INFO Getting access token... Missing details for subscription/resource group/workspace.... Available subscriptions with resource group and workspace details as below: Subscription: MY_SUBSCRIPTION ** Resource group: MY_RESOURCE_GROUP1 ** Resource group: MY_RESOURCE_GROUP2 ** Resource group: MY_RESOURCE_GROUP3 ** Workspace: MY_LOG_ANALYTICS_WORKSPACE Please re-run twigs with appropriate values for subscription, resource group and workspace. Run the command as shown below: twigs azure --azure_tenant_id AZURE_TENANT_ID --azure_application_id AZURE_APPLICATION_ID --azure_application_key AZURE_APPLICATION_KEY --azure_subscription AZURE_SUBSCRIPTION --azure_resource_group AZURE_RESOURCE_GROUP --azure_workspace AZURE_WORKSPACE [—enable_tracking_tags] It is suggested that you enable_tracking_tags, which allows you to easily identify Azure cloud instances in ThreatWorx. Note Azure cloud discovery may require some time depending on the number of VM instances in your Azure cloud setup. After discovery is complete, you can login into ThreatWorx Console to view the newly discovered assets. GCP Cloud Discovery Overview Twigs supports cloud-native discovery for Google Cloud Platform (GCP) i.e. using OS inventory management. Pre-requisites VM Manager needs to be enabled for the relevant GCP projects. VM Manager is a suite of tools provided by GCP that can be used to manage operating systems for large virtual machine (VM) fleets running Windows and Linux on Compute Engine. One of the features of the VM Manager is OS Inventory Management which enables GCP to keep an inventory of all VM instances and their associated metadata which can be used by ThreatWorx for vulnerability assessment. The easiest way to enable VM Manager for your project is to follow the instructions for any running VM on your GCP project. Enable VM Manager automatically for your project or VM Google Cloud SDK is required. Please install it on the system that will run twigs, by following instructions mentioned here for your Operating System. The SDK provides tools (like the ‘gcloud’ CLI) which are used by twigs for discovering VM instances. Steps Once VM Manager is enabled for your compute instances in GCP, you can run twigs to ingest this collected inventory into your ThreatWorx instance by following the below mentioned steps: Open a new shell / terminal Check that twigs is installed and running properly by running below command: twigs gcp -h Sign in into your Google Cloud Platform instance using gcloud CLI as described here on the box where you will be running twigs. You can run the command mentioned below: twigs gcp [--enable_tracking_tags] It is suggested that you enable_tracking_tags, which allows you to easily identify projects associated with discovered compute instances. Note GCP cloud discovery may require some time depending on the number of compute instances in your GCP cloud setup. After discovery is complete, you can login into ThreatWorx Console to view the newly discovered assets.
0
0
38
threatwatch io
Dec 12, 2022
In Getting Started With twigs
Twigs requires user details to connect to your ThreatWorx instance during the discovery process. You can use one of the three options below to initialize twigs. Option 1 twigs provides “login” and “logout” capability. This allows users to authenticate via twigs to their ThreatWorx service instance. Once logged in, users do not need to provide their details while running twigs command every time. The login command is below: twigs login Similarly to logout from twigs, you can run the command below: twigs logout This allows you to use twigs in a multi-user mode using the same installation. Option 2 Configure your environment setup (via something like .bashrc for Bash Shell) to store typical parameters required with twigs. These parameters are as follows: TW_HANDLE - User login to be used for asset ingestion. Note this user becomes the owner of the ingested assets. TW_TOKEN - API token for the specific user. You can find the value by following below steps: Login into ThreatWorx console Click on “Profile” in top menu Click on “Key Management” in left menu If you have not generated an API Key as yet, then click on “Generate New Key”, else click on “Copy to clipboard”. Note if you re-generate a new API key, then earlier key is disabled by default. TW_INSTANCE - Specify the specific TW instance provisioned for your organization. Save these as follows in your profile script (like .bashrc): export TW_HANDLE=<your login> export TW_TOKEN=<your API key> export TW_INSTANCE=<your TW instance> Next time you login, these variables will be automatically set and will help reduce the clutter in your twigs command-line. Option 3 You can also specify the three parameters as part of individual twigs runs using the --token , --instance and --handle options on the command line. Order of Precedence If all three options are provided then the order or precedence is as follows, Option 3 ( First Preference ) Option 1 ( Second Preference ) Option 2 ( Third Preference )
0
0
45
threatwatch io
Dec 12, 2022
In Getting Started With twigs
Twigs provides a bunch of common options i.e. irrespective of what type of assets are being discovered. These common options are optional as seen below: twigs [-h] [-v] [--handle HANDLE] [--token TOKEN] [--instance INSTANCE] [--run_id RUN_ID][--location LOCATION] [--tag_critical] [--tag TAG] [--no_auto_tags] [--apply_policy APPLY_POLICY][--sbom SBOM] [--no_scan] Here is a quick description of these options: run_id – Specify an unique identifier for this twigs run. Note – this identifier will help you group multiple runs for a specific type of discovery together and these will be shown together in the I3 Portal. If you don’t specify a “run_id”, then discovery “mode” will be used by default. location – Specify location for discovered asset(s) tag_critical – Mark the asset(s) as business critical tag – You can use this option multiple times on the command line to add multiple tags to the asset(s) no_auto_tags – Disable auto tagging of assets with standard classification tags. Only user specified tags will be applied. apply_policy – You can use this option to specify name of a policy when you use twigs in your CI/CD pipeline to make policy based decisions like fail the build if any “DoNow” priority vulnerability impacts are discovered or any strong copyleft violations are found. For multiple policies, specify a comma-separated list of policy names. sbom – Use this option to specify path to a SBOM file to save the asset(s) no_scan – Indicate that you don’t wish to start a vulnerability assessment for the discovered asset(s) email_report – Once the vulnerability assessment is done, then you will automatically be emailed a copy of the vulnerability assessment report. schedule – Run this twigs command at specified schedule (in crontab format) quiet – Do not display informational messages on the console during twigs run
0
0
26
threatwatch io
Dec 12, 2022
In Getting Started With twigs
Twigs cloud native images are currently available for AWS, docker hub and Digital Ocean marketplaces. These images are free to download and use and are built and maintained by ThreatWorx to contain all the requirements and dependencies of twigs including any native CLI or other open source dependencies. Please check for updates in the section as we add such images for more cloud vendors and their marketplaces. Twigs on docker hub Twigs on AWS Marketplace Twigs on Digital Ocean Marketplace For twigs docker it is necessary to have access to docker service inside the twigs instance. This can be easily accomplished by passing the docker unix socket to the instance. This can be accomplished by a a command like this while starting the twigs docker instance: docker run -it -v /var/run/docker.sock:/var/run/docker.sock threatworx/twigs:latest The docker image already contains the other dependencies required for docker image discovery.
0
0
13
threatwatch io
Dec 12, 2022
In Getting Started With twigs
Twigs can discover various classes of assets as below: Cloud assets from AWS, Azure and GCP Docker containers Import assets from a file (PDF or JSON) Servers, desktops, laptops etc. Source code repository Ingest existing assets details from CMDB like ServiceNow, etc. Twigs supports top platforms/OS as below: RedHat CentOS Ubuntu Debian Amazon Linux Windows Mac OS
0
0
9
threatwatch io
Dec 12, 2022
In Getting Started With twigs
Twigs command-line usage Please refer to twigs documentation for command-line usage Twigs source code Twigs is open source. If you are interested in going through the source code of twigs, you can find it here. Twigs is a python-based package and can be installed using ‘pip’ as below: sudo pip install twigs Note: It is recommended to use virtual environments to create isolated Python environments and reduce dependency conflicts. Please use the following commands to create new virtual environment depending on your python version and install twigs: # Python 3.x python3 -m venv twigs_env cd twigs_env source bin/activate pip3 install twigs # Python 2.7 python -m virtualenv –python=/usr/bin/python2.7 twigs_env_2_7 cd twigs_env_2_7 source bin/activate pip install twigs
0
0
18

threatwatch io

Admin
More actions
bottom of page