Skip to content

Chatbot detector flags legitimate code identifiers as secrets #4628

@rootranjan

Description

@rootranjan

Please review the Community Note before submitting

TruffleHog Version

3.92.4

Trace Output

Not applicable - this is a false positive issue, not a crash or error. The detector is working but producing incorrect results.

Expected Behavior

The Chatbot detector should not flag legitimate code identifiers (variable names, class names, schema names) as potential secrets. It should only detect actual Chatbot API keys, which typically:

  • Contain a mix of letters, numbers, and possibly underscores
  • Have higher entropy (random character distribution)
  • Don't follow common code naming conventions

Actual Behavior

The Chatbot detector flags any 32-character alphanumeric string near the keyword "chatbot" as a potential secret, including:

  • PascalCase/CamelCase variable names (e.g., internalFeatureNavigationManager)
  • Snake_case identifiers (e.g., analytics_abc_meltsys_adachatbot)
  • Class names, function names, and configuration keys

Example False Positives:

  1. PascalCase Variable Name:
class DefaultNavigationManager {
    private let internalFeatureNavigationManager: any InternalFeatureNavigating
    // ...
}

Detected: internalFeatureNavigationManager (32 chars) - This is a variable name, not a secret.

  1. Snake_case Schema Name:
sources:
  - name: analytics_abc_meltsys_adachatbot
    schema: analytics_abc_meltsys_adachatbot

Detected: analytics_abc_meltsys_adachatbot (32 chars) - This is a schema name, not a secret.

Steps to Reproduce

  1. Create a test file containing code with a 32-character identifier near the word "chatbot"
  2. Run: trufflehog filesystem --no-update <test-file>
  3. Observe false positive detection

Test Case 1 - PascalCase Variable:

// File: test.swift
class Manager {
    private let internalFeatureNavigationManager: any Navigator
    func openChatBotPage() {
        internalFeatureNavigationManager.gotoFlow(...)
    }
}

Test Case 2 - Snake_case Identifier:

# File: config.yaml
sources:
  - name: analytics_abc_meltsys_adachatbot
    schema: analytics_abc_meltsys_adachatbot

Environment

  • OS: macOS
  • Version: 3.92.4

Additional Context

The detector currently uses this pattern:

keyPat = regexp.MustCompile(detectors.PrefixRegex([]string{"chatbot"}) + `\b([a-zA-Z0-9_]{32})\b`)

This pattern is too broad and matches legitimate code identifiers.

Proposed Solution:
Add false positive filters to exclude:

  1. Only letters (no digits) - Real API keys typically contain digits
  2. Snake_case patterns - Common for variable/table names (^[a-z]+(_[a-z]+)+$)
  3. CamelCase/PascalCase patterns - Common for code identifiers
    • camelCase: ^[a-z]+([A-Z][a-z]+)+$
    • PascalCase: ^([A-Z][a-z]+)+$

Expected Outcome After Fix:

  • internalFeatureNavigationManagerFiltered (PascalCase, only letters)
  • analytics_abc_meltsys_adachatbotFiltered (snake_case pattern)
  • ✅ Real Chatbot API keys with digits → Still detected

Impact:

  • Creates noise in scan results
  • Requires manual review and filtering
  • Reduces trust in the detector's accuracy
  • Wastes verification API calls

References

  • None

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions