Testing Guide
Comprehensive testing guide covering unit tests with Vitest and end-to-end testing with Playwright for SonicJS.
Overview
Testing Philosophy
SonicJS follows a comprehensive testing approach:
- Unit Tests - Fast, isolated tests for individual functions and services
- End-to-End Tests - Browser-based tests for critical user journeys and workflows
Test Coverage Goals
- 90% minimum code coverage for core business logic
- All API endpoints have E2E test coverage
- Key user workflows have comprehensive test coverage
- Plugin functionality is thoroughly tested
Current Coverage Status
As of the latest test run:
📊
Overall Coverage
90.86% code coverage
✅
Total Tests
684 passing tests
📁
Test Files
26 test files
🎯
Statements
90.86% coverage
🔀
Branches
90.34% coverage
⚙️
Functions
96.23% coverage
Testing Stack
Core Testing Tools
⚡
Vitest
Fast, Vite-native test runner with excellent TypeScript support
🎭
Playwright
Reliable cross-browser testing with powerful debugging
📈
@vitest/coverage-v8
Fast, accurate code coverage using V8's built-in coverage
Why These Tools?
- Vitest - Fast, Vite-native test runner with excellent TypeScript support
- Playwright - Reliable cross-browser testing with powerful debugging capabilities
- Coverage-v8 - Fast, accurate code coverage using V8's built-in coverage
Setup and Installation
Prerequisites
Install Dependencies
# Install dependencies
npm install
# Install Playwright browsers (first time only)
npx playwright install
Configuration Files
The project includes pre-configured test setups:
vitest.config.ts- Vitest configurationplaywright.config.ts- Playwright configurationtests/e2e/utils/test-helpers.ts- Shared test utilities
Unit Testing with Vitest
Vitest Configuration
Vitest Config
// vitest.config.ts
import { defineConfig } from 'vitest/config'
export default defineConfig({
test: {
globals: true,
environment: 'node',
include: ['src/**/*.{test,spec}.{js,mjs,cjs,ts,mts,cts,jsx,tsx}'],
exclude: ['node_modules', 'dist', '.next'],
coverage: {
provider: 'v8',
reporter: ['text', 'json', 'html'],
include: ['src/**/*.{js,ts}'],
exclude: [
'src/**/*.{test,spec}.{js,ts}',
'src/**/*.d.ts',
'src/scripts/**',
'src/templates/**'
],
thresholds: {
global: {
branches: 90,
functions: 90,
lines: 90,
statements: 90
}
}
}
},
})
Real-World Unit Test Example
Cache Plugin Tests
import { describe, it, expect, beforeEach, vi } from 'vitest'
import {
CacheService,
createCacheService,
getCacheService,
clearAllCaches,
getAllCacheStats
} from '../services/cache.js'
import {
CACHE_CONFIGS,
getCacheConfig,
generateCacheKey,
parseCacheKey,
hashQueryParams,
createCachePattern
} from '../services/cache-config.js'
describe('CacheConfig', () => {
it('should have predefined cache configurations', () => {
expect(CACHE_CONFIGS.content).toBeDefined()
expect(CACHE_CONFIGS.user).toBeDefined()
expect(CACHE_CONFIGS.config).toBeDefined()
expect(CACHE_CONFIGS.media).toBeDefined()
})
it('should generate cache key with correct format', () => {
const key = generateCacheKey('content', 'post', '123', 'v1')
expect(key).toBe('content:post:123:v1')
})
it('should parse cache key correctly', () => {
const key = 'content:post:123:v1'
const parsed = parseCacheKey(key)
expect(parsed).toBeDefined()
expect(parsed?.namespace).toBe('content')
expect(parsed?.type).toBe('post')
expect(parsed?.identifier).toBe('123')
expect(parsed?.version).toBe('v1')
})
it('should hash query parameters consistently', () => {
const params1 = { limit: 10, offset: 0, sort: 'asc' }
const params2 = { offset: 0, limit: 10, sort: 'asc' }
const hash1 = hashQueryParams(params1)
const hash2 = hashQueryParams(params2)
expect(hash1).toBe(hash2) // Order shouldn't matter
})
})
describe('CacheService - Basic Operations', () => {
let cache: CacheService
beforeEach(() => {
const config = {
ttl: 60,
kvEnabled: false,
memoryEnabled: true,
namespace: 'test',
invalidateOn: [],
version: 'v1'
}
cache = createCacheService(config)
})
it('should set and get value from cache', async () => {
await cache.set('test:key', 'value')
const result = await cache.get('test:key')
expect(result).toBe('value')
})
it('should return null for non-existent key', async () => {
const result = await cache.get('non-existent')
expect(result).toBeNull()
})
it('should delete value from cache', async () => {
await cache.set('test:key', 'value')
await cache.delete('test:key')
const result = await cache.get('test:key')
expect(result).toBeNull()
})
})
describe('CacheService - TTL and Expiration', () => {
let cache: CacheService
beforeEach(() => {
const config = {
ttl: 1, // 1 second TTL for testing
kvEnabled: false,
memoryEnabled: true,
namespace: 'test',
invalidateOn: [],
version: 'v1'
}
cache = createCacheService(config)
})
it('should expire entries after TTL', async () => {
await cache.set('test:key', 'value')
// Wait for expiration
await new Promise(resolve => setTimeout(resolve, 1100))
const result = await cache.get('test:key')
expect(result).toBeNull()
})
it('should allow custom TTL per entry', async () => {
await cache.set('test:key', 'value', { ttl: 10 }) // 10 second TTL
// Entry should still be there after 1 second
await new Promise(resolve => setTimeout(resolve, 1100))
const result = await cache.get('test:key')
expect(result).toBe('value')
})
})
Unit Testing Patterns
Testing with Mocks
import { vi } from 'vitest'
describe('Service with Dependencies', () => {
it('should not call fetcher when value is cached', async () => {
await cache.set('test:key', 'cached-value')
const fetcher = vi.fn(async () => 'fetched-value')
const result = await cache.getOrSet('test:key', fetcher)
expect(result).toBe('cached-value')
expect(fetcher).not.toHaveBeenCalled()
})
})
End-to-End Testing with Playwright
Playwright Configuration
Playwright Config
// playwright.config.ts
import { defineConfig, devices } from '@playwright/test'
export default defineConfig({
testDir: './tests/e2e',
fullyParallel: true,
forbidOnly: !!process.env.CI,
retries: process.env.CI ? 2 : 0,
workers: process.env.CI ? 1 : undefined,
reporter: 'html',
globalSetup: require.resolve('./tests/e2e/global-setup.ts'),
globalTeardown: require.resolve('./tests/e2e/global-teardown.ts'),
use: {
baseURL: 'http://localhost:8787',
trace: 'on-first-retry',
screenshot: 'only-on-failure',
video: 'retain-on-failure',
},
projects: [
{
name: 'chromium',
use: { ...devices['Desktop Chrome'] },
},
],
webServer: {
command: 'npm run dev',
url: 'http://localhost:8787',
reuseExistingServer: !process.env.CI,
timeout: 120 * 1000,
},
})
Health Check Tests
Health Check Tests
import { test, expect } from '@playwright/test'
import { checkAPIHealth } from './utils/test-helpers'
test.describe('Health Checks', () => {
test('API health endpoint should return running status', async ({ page }) => {
const health = await checkAPIHealth(page)
expect(health).toHaveProperty('name', 'SonicJS AI')
expect(health).toHaveProperty('version', '0.1.0')
expect(health).toHaveProperty('status', 'running')
expect(health).toHaveProperty('timestamp')
})
test('Home page should redirect to login', async ({ page }) => {
const response = await page.goto('/')
expect(response?.status()).toBe(200)
// Should redirect to login page
await page.waitForURL(/\/auth\/login/)
// Verify we're on the login page
expect(page.url()).toContain('/auth/login')
await expect(page.locator('h2')).toContainText('Welcome Back')
})
test('Admin routes should require authentication', async ({ page }) => {
// Try to access admin without auth
await page.goto('/admin')
// Should redirect to login
await page.waitForURL(/\/auth\/login/)
// Verify error message is shown
await expect(page.locator('.bg-error\\/10')).toContainText(
'Please login to access the admin area'
)
})
})
Authentication Tests
Authentication Tests
import { test, expect } from '@playwright/test'
import { loginAsAdmin, logout, isAuthenticated, ADMIN_CREDENTIALS } from './utils/test-helpers'
test.describe('Authentication', () => {
test.beforeEach(async ({ page }) => {
await logout(page)
})
test('should display login form', async ({ page }) => {
await page.goto('/auth/login')
await expect(page.locator('h2')).toContainText('Welcome Back')
await expect(page.locator('[name="email"]')).toBeVisible()
await expect(page.locator('[name="password"]')).toBeVisible()
await expect(page.locator('button[type="submit"]')).toBeVisible()
})
test('should login successfully with valid credentials', async ({ page }) => {
await loginAsAdmin(page)
// Should be on admin dashboard
await expect(page).toHaveURL('/admin')
await expect(page.locator('nav').first()).toBeVisible()
})
test('should show error with invalid credentials', async ({ page }) => {
await page.goto('/auth/login')
await page.fill('[name="email"]', 'invalid@email.com')
await page.fill('[name="password"]', 'wrongpassword')
await page.click('button[type="submit"]')
// Should show error message
await expect(page.locator('.error, .bg-red-100')).toBeVisible()
})
test('should maintain session across page reloads', async ({ page }) => {
await loginAsAdmin(page)
await page.reload()
// Should still be authenticated
await expect(page).toHaveURL('/admin')
await expect(await isAuthenticated(page)).toBe(true)
})
})
Content Management Tests
Content Tests
import { test, expect } from '@playwright/test'
import {
loginAsAdmin,
navigateToAdminSection,
waitForHTMX,
ensureTestContentExists
} from './utils/test-helpers'
test.describe('Content Management', () => {
test.beforeEach(async ({ page }) => {
await loginAsAdmin(page)
await ensureTestContentExists(page)
await navigateToAdminSection(page, 'content')
})
test('should display content list', async ({ page }) => {
await expect(page.locator('h1').first()).toContainText('Content Management')
// Should have filter dropdowns
await expect(page.locator('select[name="model"]')).toBeVisible()
await expect(page.locator('select[name="status"]')).toBeVisible()
})
test('should filter content by status', async ({ page }) => {
// Filter by published status
await page.selectOption('select[name="status"]', 'published')
// Wait for HTMX to update the content
await waitForHTMX(page)
const table = page.locator('table')
const hasTable = await table.count() > 0
if (hasTable) {
const publishedRows = page.locator('tr').filter({ hasText: 'published' })
const rowCount = await publishedRows.count()
expect(rowCount).toBeGreaterThanOrEqual(0)
}
})
test('should navigate to new content form', async ({ page }) => {
await page.click('a[href="/admin/content/new"]')
await page.waitForURL('/admin/content/new', { timeout: 10000 })
// Should show collection selection page
await expect(page.locator('h1')).toContainText('Create New Content')
await expect(page.locator('text=Select a collection to create content in:')).toBeVisible()
// Should have at least one collection to select
const collectionLinks = page.locator('a[href^="/admin/content/new?collection="]')
const count = await collectionLinks.count()
expect(count).toBeGreaterThan(0)
})
})
API Testing with Playwright
API Tests
import { test, expect } from '@playwright/test'
test.describe('API Endpoints', () => {
test('should return health check', async ({ request }) => {
const response = await request.get('/health')
expect(response.ok()).toBeTruthy()
const health = await response.json()
expect(health).toHaveProperty('name', 'SonicJS AI')
expect(health).toHaveProperty('version', '0.1.0')
expect(health).toHaveProperty('status', 'running')
})
test('should return OpenAPI spec', async ({ request }) => {
const response = await request.get('/api')
expect(response.ok()).toBeTruthy()
const spec = await response.json()
expect(spec).toHaveProperty('openapi')
expect(spec).toHaveProperty('info')
expect(spec).toHaveProperty('paths')
})
test('should handle SQL injection attempts safely', async ({ request }) => {
const sqlInjectionAttempts = [
"'; DROP TABLE collections; --",
"' OR '1'='1",
"'; SELECT * FROM users; --",
]
for (const injection of sqlInjectionAttempts) {
const response = await request.get(
`/api/collections/${encodeURIComponent(injection)}/content`
)
// Should safely return 404, not expose database errors
expect(response.status()).toBe(404)
const data = await response.json()
expect(data.error).toBe('Collection not found')
// Should not expose SQL error messages
expect(data.error).not.toContain('SQL')
expect(data.error).not.toContain('database')
}
})
})
Running Tests
Unit Tests
Unit Test Commands
# Run all unit tests
npm test
# Run tests in watch mode
npm run test:watch
# Run with coverage
npm run test:cov
# Run with coverage in watch mode
npm run test:cov:watch
# Run with coverage and UI
npm run test:cov:ui
E2E Tests
E2E Test Commands
# Run all E2E tests
npm run test:e2e
# Run E2E tests with UI mode
npm run test:e2e:ui
# Run specific test file
npx playwright test tests/e2e/02-authentication.spec.ts
# Run tests in headed mode (see browser)
npx playwright test --headed
# Run tests in debug mode
npx playwright test --debug
Coverage Reporting
Viewing Coverage Reports
Coverage Commands
# Generate coverage report
npm run test:cov
# Coverage files are generated in:
# - coverage/index.html (HTML report)
# - coverage/coverage-final.json (JSON report)
Coverage Thresholds
The project enforces minimum coverage thresholds:
thresholds: {
global: {
branches: 90,
functions: 90,
lines: 90,
statements: 90
}
}
Recent Coverage Improvements:
The project recently increased coverage from 87% to over 90% by adding comprehensive tests for:
- Media storage operations - 92.96%
- Image optimization - 91.74%
- Cache plugin functionality - extensive coverage
- Core services (CDN, notifications, scheduler, workflow) - all >93%
Testing Plugins
Plugin Test Structure
Plugins include their own test files:
src/plugins/cache/
├── services/
│ ├── cache.ts
│ └── cache-config.ts
└── tests/
└── cache.test.ts
Example Plugin Test
Plugin Tests
describe('CacheService - Batch Operations', () => {
let cache: CacheService
beforeEach(() => {
cache = createCacheService(CACHE_CONFIGS.content!)
})
it('should get multiple values at once', async () => {
await cache.set('key1', 'value1')
await cache.set('key2', 'value2')
await cache.set('key3', 'value3')
const results = await cache.getMany(['key1', 'key2', 'key3', 'key4'])
expect(results.size).toBe(3)
expect(results.get('key1')).toBe('value1')
expect(results.get('key2')).toBe('value2')
expect(results.has('key4')).toBe(false)
})
it('should set multiple values at once', async () => {
await cache.setMany([
{ key: 'key1', value: 'value1' },
{ key: 'key2', value: 'value2' },
{ key: 'key3', value: 'value3' }
])
const value1 = await cache.get('key1')
const value2 = await cache.get('key2')
expect(value1).toBe('value1')
expect(value2).toBe('value2')
})
})
Test Helpers and Utilities
Common Test Helpers
Test Helpers
// Authentication
export const ADMIN_CREDENTIALS = {
email: 'admin@sonicjs.com',
password: 'admin123'
}
export async function loginAsAdmin(page: Page) {
await ensureAdminUserExists(page)
await page.goto('/auth/login')
await page.fill('[name="email"]', ADMIN_CREDENTIALS.email)
await page.fill('[name="password"]', ADMIN_CREDENTIALS.password)
await page.click('button[type="submit"]')
await expect(page.locator('#form-response .bg-green-100')).toBeVisible()
await page.waitForURL('/admin', { timeout: 15000 })
}
// Navigation
export async function navigateToAdminSection(
page: Page,
section: 'collections' | 'content' | 'media' | 'users'
) {
await page.click(`a[href="/admin/${section}"]`)
await page.waitForURL(`/admin/${section}`)
}
// HTMX Support
export async function waitForHTMX(page: Page) {
try {
await page.waitForLoadState('networkidle', { timeout: 5000 })
} catch {
await page.waitForTimeout(1000)
}
}
// API Health Check
export async function checkAPIHealth(page: Page) {
const response = await page.request.get('/health')
expect(response.ok()).toBeTruthy()
const health = await response.json()
expect(health.status).toBe('running')
return health
}
Best Practices
1. Test Organization
- Keep tests close to code - Unit tests live alongside the code they test
- Logical grouping - Use
describeblocks to organize related tests - Clear naming - Test names should describe what is being tested and expected outcome
Test Organization
describe('CacheService - Pattern Invalidation', () => {
it('should invalidate entries matching pattern', async () => {
// Test implementation
})
it('should not invalidate entries that do not match pattern', async () => {
// Test implementation
})
})
2. Test Independence
- Each test should be independent and not rely on other tests
- Use
beforeEachto set up fresh state - Clean up after tests when necessary
Test Independence
describe('My Feature', () => {
beforeEach(() => {
// Set up fresh state for each test
cache = createCacheService(config)
})
afterEach(async () => {
// Clean up if needed
await cache.clear()
})
})
3. Async Testing
- Always use
async/awaitfor asynchronous operations - Don't forget to
awaitpromises in tests
Async Testing
// Good
it('should fetch data', async () => {
const result = await fetchData()
expect(result).toBeDefined()
})
// Bad - missing await
it('should fetch data', async () => {
const result = fetchData() // Missing await!
expect(result).toBeDefined() // Will fail
})
4. Playwright Best Practices
- Use test helpers - Create reusable functions for common operations
- Wait for elements - Use Playwright's built-in waiting mechanisms
- Avoid fixed timeouts - Prefer
waitForSelectoroverwaitForTimeout - Handle HTMX - Use the
waitForHTMXhelper for dynamic updates
Testing Best Practices
- Always define TypeScript interfaces for test data
- Break down complex tests into smaller, focused tests
- Use fixtures and factories for consistent test data
- Test both success and failure cases
- Verify error messages and status codes
- Keep tests fast by mocking external dependencies
- Run tests in CI/CD pipeline before deployment