query-orchestration/test/integration/README.md

Admin User Management Integration Tests

This directory contains integration tests for admin user management functionality that use real AWS Cognito and Permit.io APIs (no mocks).

Overview

These tests validate the complete user management workflow including:

• User creation in AWS Cognito and Permit.io
• User retrieval and attribute updates
• User enable/disable operations
• User deletion from both systems
• Role assignment and unassignment in Permit.io
• User listing with pagination
• Idempotent operations

Why Integration Tests Are Separate

The admin user management code is excluded from standard CI/CD coverage requirements because:

1. Free tier LocalStack doesn't support AWS Cognito
2. Integration tests require real AWS credentials
3. Integration tests require real Permit.io API access
4. Tests create and modify real user accounts
5. Tests should not run automatically in CI/CD pipelines

These tests are designed for manual execution during development and pre-production validation.

Prerequisites

AWS Requirements

use task aws:login then eval $(aws configure export-credentials --profile aarete --format env) to set fresh credentials for the test to use.

1. AWS Account with access to Cognito User Pool

2. IAM User or Role with the following permissions:

   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Effect": "Allow",
         "Action": [
           "cognito-idp:AdminCreateUser",
           "cognito-idp:AdminDeleteUser",
           "cognito-idp:AdminGetUser",
           "cognito-idp:AdminUpdateUserAttributes",
           "cognito-idp:AdminDisableUser",
           "cognito-idp:AdminEnableUser",
           "cognito-idp:ListUsers"
         ],
         "Resource": "arn:aws:cognito-idp:REGION:ACCOUNT_ID:userpool/POOL_ID"
       }
     ]
   }
   

3. Cognito User Pool ID - Get this from AWS Console or CLI

Permit.io Requirements

1. Permit.io Account with API access
2. API Key with permissions to:
   • Create users
   • Delete users
   • Assign/unassign roles
   • Read user roles
3. Project ID and Environment ID from Permit.io dashboard
4. Roles configured in Permit.io matching your permit_policies.yaml:
   • super_admin
   • user_admin
   • auditor
   • client_user

Setup Instructions

Step 1: Create Environment File

cd test/integration
cp .env.example .env

Step 2: Configure Credentials

Edit test/integration/.env and fill in your actual credentials:

# AWS Configuration
AWS_REGION=us-east-1
AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
COGNITO_USER_POOL_ID=us-east-1_AbCdEfGhI

# Suppress welcome emails to avoid AWS Cognito's 50 emails/day limit
COGNITO_SUPPRESS_EMAILS=true

# Skip Permit.io cleanup (set to true to inspect users after tests)
SKIP_PERMITIO_CLEANUP=false

# Permit.io Configuration
PERMIT_IO_API_KEY=permit_key_1234567890abcdef
PERMIT_IO_PROJECT_ID=my-project-123
PERMIT_IO_ENV_ID=dev
PERMIT_IO_TENANT=default

# Test Configuration
TEST_USER_EMAIL_PREFIX=int-test-
TEST_USER_DOMAIN=example.com

IMPORTANT: Never commit the .env file to git! It contains sensitive credentials.

Step 3: Verify Permit.io Roles

Note: The test suite automatically loads environment variables from test/integration/.env when you run the tests. You do NOT need to manually source the file.

Ensure the following roles exist in your Permit.io environment (as defined in cmd/auth_related/permit.setup/permit_policies.yaml):

• super_admin
• user_admin
• auditor
• client_user

You can create roles through the Permit.io dashboard, API, or use the setup tool at cmd/auth_related/permit.setup/.

Running the Tests

Run All Integration Tests (13 total)

go test -tags=integration -v ./test/integration/...

This runs:

• 8 integrated tests (Cognito + Permit.io)
• 5 Permit.io-only tests (no Cognito calls)

Run Only Integrated Tests (Cognito + Permit.io)

go test -tags=integration -v -run 'TestSuite/Test[^P]' ./test/integration/...

This excludes tests starting with TestPermitIO_.

Run Only Permit.io Tests (No Cognito)

go test -tags=integration -v -run 'TestSuite/TestPermitIO' ./test/integration/...

Useful when you've hit AWS Cognito's email limit or want to test Permit.io in isolation.

Run a Specific Test

go test -tags=integration -v -run TestSuite/TestCreateUser ./test/integration/...

Run via Task Command

Add this task to your Taskfile.yml:

integration:admin:
  desc: "Run admin user management integration tests (requires real AWS/Permit.io credentials)"
  dir: test/integration
  cmds:
    - |
      if [ ! -f .env ]; then
        echo "❌ Error: test/integration/.env not found"
        echo "Copy .env.example to .env and configure your credentials"
        exit 1
      fi
    - go test -tags=integration -v ./...

Then run:

task integration:admin

Test Coverage

The test suite includes 13 integration tests organized into two categories:

Integrated Tests (8 tests - Cognito + Permit.io)

These tests validate the complete user management workflow across both systems:

Test	Description
TestCreateUser	Creates a new user in both Cognito and Permit.io
TestCreateUserIdempotent	Verifies idempotent user creation behavior
TestGetUser	Retrieves an existing user's details from Cognito
TestUpdateUserAttributes	Updates user's first name and last name in Cognito
TestDisableEnableUser	Disables and re-enables a user account in Cognito
TestDeleteUser	Deletes user from both Cognito and Permit.io
TestListUsers	Lists users with pagination support from Cognito
TestRoleAssignment	Assigns and unassigns roles in Permit.io for Cognito user

Note: These tests require AWS credentials and will send emails unless COGNITO_SUPPRESS_EMAILS=true.

Permit.io-Only Tests (5 tests - No Cognito)

These tests validate Permit.io functionality in isolation using generated UUIDs:

Test	Description
TestPermitIO_CreateAndGetUser	Creates user in Permit.io and verifies it exists
TestPermitIO_AssignMultipleRoles	Assigns/unassigns multiple roles (auditor, client_user)
TestPermitIO_DeleteUser	Deletes user and verifies with 404 response
TestPermitIO_UserLifecycle	Complete 9-step user lifecycle (create → roles → delete)
TestPermitIO_IdempotentOperations	Tests duplicate user/role operations are idempotent

Note: These tests do NOT call AWS Cognito and can run unlimited times per day.

Test User Management

Automatic Cleanup

The test suite implements automatic cleanup in the TearDownSuite method:

• All test users are tracked during creation
• After all tests complete, users are automatically deleted from both Cognito and Permit.io
• If cleanup fails, warnings are logged but tests still pass
• Optional: Set SKIP_PERMITIO_CLEANUP=true to preserve Permit.io users for manual inspection (see "Manual Inspection of Test Users" section)

Manual Cleanup

If tests are interrupted and cleanup doesn't run:

# List users in Cognito
aws cognito-idp list-users --user-pool-id YOUR_POOL_ID

# Delete a user from Cognito
aws cognito-idp admin-delete-user \
  --user-pool-id YOUR_POOL_ID \
  --username user@example.com

# Delete from Permit.io (requires API call)
curl -X DELETE \
  "https://api.permit.io/v2/facts/PROJECT_ID/ENV_ID/users/SUBJECT_ID" \
  -H "Authorization: Bearer YOUR_API_KEY"

Test Email Format

Test users are created with emails in this format:

{PREFIX}{TEST_NAME}-{TIMESTAMP}@{DOMAIN}

Examples:

• Integrated test: int-test-create-1697123456@example.com
• Permit.io test: int-test-permitio-create-1697123456@example.com

This makes it easy to identify and clean up test users. You can customize the prefix and domain using environment variables:

• TEST_USER_EMAIL_PREFIX (default: int-test-)
• TEST_USER_DOMAIN (default: example.com)

AWS Cognito Email Limits

AWS Cognito has a daily limit of 50 emails when using the default email service. This limit:

• Applies to welcome emails sent during user creation
• Resets daily at 09:00 UTC
• Is shared across all user pools in your AWS account
• Cannot be increased without configuring Amazon SES

Email Suppression for Testing

To avoid hitting this limit during integration testing, the test suite uses the COGNITO_SUPPRESS_EMAILS=true environment variable. When set:

• ✅ No welcome emails are sent to test users
• ✅ Users are still created successfully with email_verified: true
• ✅ Tests can run unlimited times per day
• ✅ All user management operations work normally

This is the recommended approach for integration tests.

If You Need to Test Email Delivery

For production environments or when you need to test actual email delivery:

1. Configure Amazon SES with your Cognito User Pool
2. Move SES out of sandbox mode (requires AWS support ticket, 24hr response)
3. Email limits increase to 50,000/day in SES production mode

See AWS Cognito Email Settings Documentation for setup instructions.

Manual Inspection of Test Users

To inspect users created during tests in the Permit.io dashboard:

1. Set the skip cleanup flag in test/integration/.env:

   SKIP_PERMITIO_CLEANUP=true
   

2. Run the Permit.io-only tests:

   go test -tags=integration -v -run 'TestSuite/TestPermitIO' ./test/integration/...
   

3. Check test output for user details:

   🔵 Permit.io-only user (SubjectID: cbfc1ba6-81a1-40e1-8e4d-14950737c08f)
   ✅ User found in Permit.io:
      - Roles: [auditor]
   ⏭️  Skipping Permit.io cleanup - user left for manual inspection
   

4. Log into Permit.io dashboard and inspect the users with the SubjectIDs from the test output

5. Clean up manually when done:

   • In Permit.io UI: Delete the test users
   • Or re-run tests with SKIP_PERMITIO_CLEANUP=false

Note: Cognito users (if any) are still cleaned up automatically to avoid AWS resource charges. Only Permit.io users are preserved when this flag is set.

Troubleshooting

Error: Required environment variable not set

Solution: Ensure all required variables are set in test/integration/.env

Error: User pool not found

Solution: Verify COGNITO_USER_POOL_ID is correct and your AWS credentials have access

Error: Permit.io API authentication failed

Solution:

• Verify PERMIT_IO_API_KEY is correct
• Check that the API key has not expired
• Ensure the API key has correct permissions

Error: Role not found in Permit.io

Solution: Create the required roles in your Permit.io environment or update test to use existing roles

Error: LimitExceededException - Exceeded daily email limit

api error LimitExceededException: Exceeded daily email limit for the operation or the account.

Solution:

• Ensure COGNITO_SUPPRESS_EMAILS=true is set in test/integration/.env
• The email limit resets at 09:00 UTC each day
• For production use, configure Amazon SES with your user pool (see AWS Cognito Email Limits section above)

Tests hang or timeout

Solution:

• Check network connectivity to AWS and Permit.io
• Verify AWS credentials are not expired (especially for SSO/temporary credentials)
• Increase timeout if needed (edit httpClient.Timeout in test suite)

Cleanup warnings

Solution: Warnings during cleanup are usually okay (e.g., user already deleted). However, if you see many failures:

• Check AWS credentials are still valid
• Verify Permit.io API key permissions
• Manually clean up orphaned test users

CI/CD Exclusion

The admin handler code is intentionally excluded from CI/CD coverage checks:

Excluded files in scripts/tests.yml:

• api/queryAPI/adminHandlers.go
• internal/usermanagement/audit.go
• internal/usermanagement/cognito.go
• internal/usermanagement/permitio.go
• internal/usermanagement/types.go

This allows the main test suite to pass without requiring cloud credentials.

Security Considerations

1. Never commit credentials - The .env file is gitignored
2. Use test environments - Don't run against production Cognito/Permit.io
3. Limit IAM permissions - Use least privilege for test credentials
4. Rotate credentials - Regularly rotate API keys and access keys
5. Monitor test accounts - Review test user creation in CloudWatch/Permit.io logs
6. Clean up regularly - Ensure test users are deleted after runs

Future Enhancements

Potential improvements for the integration test suite:

• Add tests for concurrent user operations
• Test bulk user creation/deletion
• Add tests for edge cases (special characters in names, etc.)
• Test Cognito password reset flow
• Test MFA enrollment/verification
• Add performance benchmarks
• Test rate limiting behavior
• Add tests for invalid/malformed requests
• Test permission boundary scenarios in Permit.io
• Add chaos testing (network failures, timeouts, etc.)

Support

For issues with:

• AWS Cognito: Check AWS documentation or contact AWS Support
• Permit.io: Check Permit.io documentation or contact support
• Test suite: Open an issue in the project repository