Documentation
¶
Index ¶
- Constants
- Variables
- func GetFalsePositiveCheck(detector Detector) func(Result) (bool, string)
- func HasDigit(key string) bool
- func IsKnownFalsePositive(match string, falsePositives map[FalsePositive]struct{}, wordCheck bool) (bool, string)
- func KeyIsRandom(key string) bool
- func MustGetBenchmarkData() map[string][]byte
- func NewDetectorHttpClient(opts ...ClientOption) *http.Client
- func NewDetectorTransport(T http.RoundTripper) http.RoundTripper
- func OverrideDetectorTimeout(timeout time.Duration)
- func ParseURLAndStripPathAndParams(u string) (*url.URL, error)
- func PrefixRegex(keywords []string) string
- func RedactURL(u url.URL) string
- func StringShannonEntropy(input string) float64
- type ClientOption
- type CloudProvider
- type CustomFalsePositiveChecker
- type CustomMultiPartCredentialProvider
- type CustomResultsCleaner
- type DefaultMultiPartCredentialProvider
- type Detector
- type EndpointCustomizer
- type EndpointSetter
- func (e *EndpointSetter) Endpoints(foundEndpoints ...string) []string
- func (e *EndpointSetter) SetCloudEndpoint(url string)
- func (e *EndpointSetter) SetConfiguredEndpoints(userConfiguredEndpoints ...string) error
- func (e *EndpointSetter) UseCloudEndpoint(enabled bool)
- func (e *EndpointSetter) UseFoundEndpoints(enabled bool)
- type FalsePositive
- type MaxSecretSizeProvider
- type MultiPartCredentialProvider
- type Result
- type ResultWithMetadata
- type StartOffsetProvider
- type Versioner
Constants ¶
const DefaultResponseTimeout = 10 * time.Second
DefaultResponseTimeout is the default timeout for HTTP requests.
Variables ¶
var ( DefaultFalsePositives = map[FalsePositive]struct{}{ "example": {}, "xxxxxx": {}, "aaaaaa": {}, "abcde": {}, "00000": {}, "sample": {}, "*****": {}, } UuidFalsePositives map[FalsePositive]struct{} )
var DetectorHttpClientWithLocalAddresses *http.Client
var DetectorHttpClientWithNoLocalAddresses *http.Client
var ErrNoLocalIP = errors.New("dialing local IP addresses is not allowed")
Functions ¶
func GetFalsePositiveCheck ¶ added in v3.75.0
func IsKnownFalsePositive ¶
func IsKnownFalsePositive(match string, falsePositives map[FalsePositive]struct{}, wordCheck bool) (bool, string)
IsKnownFalsePositive returns whether a finding is (likely) a known false positive, and the reason for the detection.
Currently, this includes: english word in key or matches common example patterns. Only the secret key material should be passed into this function
func KeyIsRandom ¶
KeyIsRandom is a Low cost check to make sure that 'keys' include a number to reduce FPs. Golang doesn't support regex lookaheads, so must be done in separate calls. TODO improve checks. Shannon entropy did not work well.
func MustGetBenchmarkData ¶
func NewDetectorHttpClient ¶ added in v3.81.9
func NewDetectorHttpClient(opts ...ClientOption) *http.Client
func NewDetectorTransport ¶ added in v3.81.9
func NewDetectorTransport(T http.RoundTripper) http.RoundTripper
func OverrideDetectorTimeout ¶ added in v3.88.14
OverrideDetectorTimeout overrides the default timeout for the detector HTTP clients. It is guaranteed to only run once, subsequent calls will have no effect. This should be called before any scans are started.
func ParseURLAndStripPathAndParams ¶ added in v3.81.9
func PrefixRegex ¶
PrefixRegex ensures that at least one of the given keywords is within 40 characters of the capturing group that follows. This can help prevent false positives.
func StringShannonEntropy ¶ added in v3.60.0
Types ¶
type ClientOption ¶ added in v3.81.9
ClientOption defines a function type that modifies an http.Client.
func WithNoFollowRedirects ¶ added in v3.81.9
func WithNoFollowRedirects() ClientOption
WithNoFollowRedirects allows disabling automatic following of redirects.
func WithNoLocalIP ¶ added in v3.81.9
func WithNoLocalIP() ClientOption
func WithTimeout ¶ added in v3.81.9
func WithTimeout(timeout time.Duration) ClientOption
WithTimeout sets a timeout for the http.Client.
func WithTransport ¶ added in v3.81.9
func WithTransport(transport http.RoundTripper) ClientOption
WithTransport sets a custom transport for the http.Client.
type CloudProvider ¶ added in v3.82.4
type CloudProvider interface {
CloudEndpoint() string
}
type CustomFalsePositiveChecker ¶ added in v3.75.0
type CustomMultiPartCredentialProvider ¶ added in v3.78.1
type CustomMultiPartCredentialProvider struct {
// contains filtered or unexported fields
}
func NewCustomMultiPartCredentialProvider ¶ added in v3.78.1
func NewCustomMultiPartCredentialProvider(maxCredentialSpan int64) *CustomMultiPartCredentialProvider
NewCustomMultiPartCredentialProvider creates a new instance of CustomMultiPartCredentialProvider with the specified maximum credential span.
func (CustomMultiPartCredentialProvider) MaxCredentialSpan ¶ added in v3.78.1
func (d CustomMultiPartCredentialProvider) MaxCredentialSpan() int64
MaxCredentialSpan returns the custom maximum credential span specified during the creation of the CustomMultiPartCredentialProvider.
type CustomResultsCleaner ¶ added in v3.81.10
type CustomResultsCleaner interface { // CleanResults removes "superfluous" results from a result set (where the definition of "superfluous" is detector- // specific). CleanResults(results []Result) []Result // ShouldCleanResultsIrrespectiveOfConfiguration allows a custom cleaner to instruct the engine to ignore // user-provided configuration that controls whether results are cleaned. (User-provided configuration is not the // only factor that determines whether the engine runs cleaning logic.) ShouldCleanResultsIrrespectiveOfConfiguration() bool }
CustomResultsCleaner is an optional interface that a detector can implement to customize how its generated results are "cleaned," which is defined as removing superfluous results from those found in a given chunk. The default implementation of this logic removes all unverified results if there are any verified results, and all unverified results except for one otherwise, but this interface allows a detector to specify different logic. (This logic must be implemented outside results generation because there are circumstances under which the engine should not execute it.)
type DefaultMultiPartCredentialProvider ¶ added in v3.78.1
type DefaultMultiPartCredentialProvider struct{}
func (DefaultMultiPartCredentialProvider) MaxCredentialSpan ¶ added in v3.78.1
func (d DefaultMultiPartCredentialProvider) MaxCredentialSpan() int64
MaxCredentialSpan returns the default maximum credential span of 1024 for the DefaultMultiPartCredentialProvider.
type Detector ¶
type Detector interface { // FromData will scan bytes for results, and optionally verify them. FromData(ctx context.Context, verify bool, data []byte) ([]Result, error) // Keywords are used for efficiently pre-filtering chunks using substring operations. // Use unique identifiers that are part of the secret if you can, or the provider name. Keywords() []string // Type returns the DetectorType number from detectors.proto for the given detector. Type() detectorspb.DetectorType // Description returns a description for the result being detected Description() string }
Detector defines an interface for scanning for and verifying secrets.
type EndpointCustomizer ¶ added in v3.34.0
type EndpointCustomizer interface { SetConfiguredEndpoints(...string) error SetCloudEndpoint(string) UseCloudEndpoint(bool) UseFoundEndpoints(bool) }
EndpointCustomizer is an optional interface that a detector can implement to support verifying against user-supplied endpoints.
type EndpointSetter ¶ added in v3.34.0
type EndpointSetter struct {
// contains filtered or unexported fields
}
EndpointSetter implements a sensible default for the SetEndpoints function of the EndpointCustomizer interface. A detector can embed this struct to gain the functionality.
func (*EndpointSetter) Endpoints ¶ added in v3.34.0
func (e *EndpointSetter) Endpoints(foundEndpoints ...string) []string
func (*EndpointSetter) SetCloudEndpoint ¶ added in v3.82.4
func (e *EndpointSetter) SetCloudEndpoint(url string)
func (*EndpointSetter) SetConfiguredEndpoints ¶ added in v3.82.4
func (e *EndpointSetter) SetConfiguredEndpoints(userConfiguredEndpoints ...string) error
func (*EndpointSetter) UseCloudEndpoint ¶ added in v3.82.4
func (e *EndpointSetter) UseCloudEndpoint(enabled bool)
func (*EndpointSetter) UseFoundEndpoints ¶ added in v3.82.4
func (e *EndpointSetter) UseFoundEndpoints(enabled bool)
type FalsePositive ¶
type FalsePositive string
type MaxSecretSizeProvider ¶ added in v3.78.1
type MaxSecretSizeProvider interface {
MaxSecretSize() int64
}
MaxSecretSizeProvider is an optional interface that a detector can implement to provide a custom max size for the secret it finds.
type MultiPartCredentialProvider ¶ added in v3.78.1
type MultiPartCredentialProvider interface { // MaxCredentialSpan returns the maximum span or range of characters that the // detector should consider when searching for a multi-part credential. MaxCredentialSpan() int64 }
MultiPartCredentialProvider is an optional interface that a detector can implement to indicate its compatibility with multi-part credentials and provide the maximum secret size for the credential it finds.
type Result ¶
type Result struct { // DetectorType is the type of Detector. DetectorType detectorspb.DetectorType // DetectorName is the name of the Detector. Used for custom detectors. DetectorName string Verified bool // VerificationFromCache indicates whether this result's verification result came from the verification cache rather // than an actual remote request. VerificationFromCache bool // Raw contains the raw secret identifier data. Prefer IDs over secrets since it is used for deduping after hashing. Raw []byte // RawV2 contains the raw secret identifier that is a combination of both the ID and the secret. // This is used for secrets that are multi part and could have the same ID. Ex: AWS credentials RawV2 []byte // Redacted contains the redacted version of the raw secret identification data for display purposes. // A secret ID should be used if available. Redacted string ExtraData map[string]string StructuredData *detectorspb.StructuredData // AnalysisInfo should be set with information required for credential // analysis to run. The keys of the map are analyzer specific and // should match what is expected in the corresponding analyzer. AnalysisInfo map[string]string // contains filtered or unexported fields }
func CleanResults ¶
CleanResults returns all verified secrets, and if there are no verified secrets, just one unverified secret if there are any.
func FilterKnownFalsePositives ¶ added in v3.74.0
FilterKnownFalsePositives filters out known false positives from the results.
func FilterResultsWithEntropy ¶ added in v3.60.0
func FilterResultsWithEntropy(ctx context.Context, results []Result, entropy float64, shouldLog bool) []Result
FilterResultsWithEntropy filters out determinately unverified results that have a shannon entropy below the given value.
func (*Result) CopyVerificationInfo ¶ added in v3.88.0
CopyVerificationInfo clones verification info (status and error) from another Result struct. This is used when loading verification info from a verification cache. (A method is necessary because verification errors are not exported, to prevent the accidental storage of sensitive information in them.)
func (*Result) SetVerificationError ¶ added in v3.63.2
SetVerificationError is the only way to set a new verification error. Any sensitive values should be passed-in as secrets to be redacted.
func (*Result) VerificationError ¶ added in v3.44.0
Public accessors for the fields could also be provided if needed.
type ResultWithMetadata ¶
type ResultWithMetadata struct { // IsWordlistFalsePositive indicates whether this secret was flagged as a false positive based on a wordlist check IsWordlistFalsePositive bool // SourceMetadata contains source-specific contextual information. SourceMetadata *source_metadatapb.MetaData // SourceID is the ID of the source that the API uses to map secrets to specific sources. SourceID sources.SourceID // JobID is the ID of the job that the API uses to map secrets to specific jobs. JobID sources.JobID // SecretID is the ID of the secret, if it exists. // Only secrets that are being reverified will have a SecretID. SecretID int64 // SourceType is the type of Source. SourceType sourcespb.SourceType // SourceName is the name of the Source. SourceName string Result // Data from the sources.Chunk which this result was emitted for Data []byte // DetectorDescription is the description of the Detector. DetectorDescription string // DecoderType is the type of decoder that was used to generate this result's data. DecoderType detectorspb.DecoderType }
func CopyMetadata ¶
func CopyMetadata(chunk *sources.Chunk, result Result) ResultWithMetadata
CopyMetadata returns a detector result with included metadata from the source chunk.
type StartOffsetProvider ¶ added in v3.78.1
type StartOffsetProvider interface {
StartOffset() int64
}
StartOffsetProvider is an optional interface that a detector can implement to provide a custom start offset for the secret it finds.
Source Files
¶
Directories
¶
Path | Synopsis |
---|---|
atlassian
|
|
buildkite
|
|
captaindata
|
|
dockerhub
|
|
elevenlabs
|
|
figmapersonalaccesstoken
|
|
fullstory
|
|
github
|
|
gitlab
|
|
godaddy
|
|
hubspot_apikey
|
|
jiratoken
|
|
maxmindlicense
|
|
sentrytoken
|
|
twitter
|
|
typeform
|
|