214 lines
6.9 KiB
Markdown
214 lines
6.9 KiB
Markdown
# Dashboard WebSocket Integration Tests
|
|
|
|
This directory contains integration tests for the dashboard's WebSocket functionality, focusing on real-time synchronization between the actual server and dashboard client.
|
|
|
|
## Overview
|
|
|
|
The integration tests verify that the dashboard correctly:
|
|
- Connects to WebSocket endpoints on a real running server
|
|
- Receives and processes WebSocket messages from the actual server
|
|
- Maintains connection stability and handles reconnection
|
|
- Loads all data types correctly through the real API
|
|
|
|
## Test Architecture
|
|
|
|
### Real Integration Testing
|
|
- **Spins up actual xp_server process** with `--db` and `--port` flags
|
|
- **Uses real ApiService and DashboardProvider** - no mocked implementations
|
|
- **Tests actual WebSocket communication** between server and client
|
|
- **Supports custom databases** for testing different scenarios
|
|
|
|
### Core Files
|
|
|
|
- **`dashboard_websocket_integration_test.dart`** - Main integration test suite
|
|
- **`run_dashboard_test.sh`** - Test runner script with `--db` flag support
|
|
- **`README.md`** - This documentation
|
|
|
|
### Golden Test Databases
|
|
|
|
Golden test databases are created using `xp_server/test/create_golden_db.dart` and contain pre-populated data for specific testing scenarios.
|
|
|
|
## Running Tests
|
|
|
|
### Basic Usage
|
|
|
|
```bash
|
|
# Run with server's default database
|
|
cd xp_dashboard
|
|
./test/integration/run_dashboard_test.sh
|
|
|
|
# Or run directly with Flutter
|
|
flutter test test/integration/dashboard_websocket_integration_test.dart
|
|
```
|
|
|
|
### Using Custom Databases
|
|
|
|
```bash
|
|
# Run with a specific database file
|
|
./test/integration/run_dashboard_test.sh --db path/to/custom.db
|
|
|
|
# Run with golden test database
|
|
./test/integration/run_dashboard_test.sh --db ../xp_server/test/high_achiever.db
|
|
```
|
|
|
|
### Creating Golden Test Databases
|
|
|
|
Golden test databases are created in the server directory:
|
|
|
|
```bash
|
|
cd xp_server/test
|
|
|
|
# Create different scenario databases
|
|
dart create_golden_db.dart high_achiever high_achiever.db
|
|
dart create_golden_db.dart new_user new_user.db
|
|
dart create_golden_db.dart level_up_ready level_up.db
|
|
dart create_golden_db.dart achievement_rich achievements.db
|
|
dart create_golden_db.dart focus_master focus.db
|
|
|
|
# Then run tests with these databases
|
|
cd ../../xp_dashboard
|
|
./test/integration/run_dashboard_test.sh --db ../xp_server/test/high_achiever.db
|
|
```
|
|
|
|
## Golden Test Scenarios
|
|
|
|
### `high_achiever`
|
|
- **Level**: 8-9 (2500+ XP)
|
|
- **Features**: Multiple achievements, extensive activity history, many focus sessions
|
|
- **Use Case**: Testing high-level user experience, complex data handling
|
|
|
|
### `new_user`
|
|
- **Level**: 1 (50 XP)
|
|
- **Features**: Minimal data, first achievement, basic activities
|
|
- **Use Case**: Testing new user onboarding, minimal data scenarios
|
|
|
|
### `level_up_ready`
|
|
- **Level**: 4 (850 XP, close to level 5)
|
|
- **Features**: Just below level threshold, good activity mix
|
|
- **Use Case**: Testing level-up notifications and threshold behavior
|
|
|
|
### `achievement_rich`
|
|
- **Level**: 5 (1200 XP)
|
|
- **Features**: Many recent achievements (last 5 minutes to 2 hours)
|
|
- **Use Case**: Testing achievement notification handling, recent achievement display
|
|
|
|
### `focus_master`
|
|
- **Level**: 7 (1800 XP)
|
|
- **Features**: 15 focus sessions, 9 hours focus time, focus-related achievements
|
|
- **Use Case**: Testing focus session workflows, high focus activity scenarios
|
|
|
|
## Test Scenarios
|
|
|
|
The integration test covers these key scenarios:
|
|
|
|
1. **WebSocket Connection & Data Loading**
|
|
- Server process startup and connection
|
|
- Initial data synchronization
|
|
- Connection state verification
|
|
|
|
2. **Connection Management**
|
|
- Connection maintenance over time
|
|
- Manual disconnection and reconnection
|
|
- Heartbeat/ping-pong handling
|
|
|
|
3. **Real-time Updates**
|
|
- Listening for server-generated updates
|
|
- State change detection
|
|
- Update processing verification
|
|
|
|
4. **Data Loading**
|
|
- Stats loading through real API
|
|
- Achievements, activities, focus sessions
|
|
- XP breakdown and classifications
|
|
- Error handling for API calls
|
|
|
|
## Server Configuration
|
|
|
|
The test automatically:
|
|
- Starts xp_server with random port (8000-9000 range) to avoid conflicts
|
|
- Passes `--db` flag to server if custom database specified
|
|
- Waits for server startup confirmation
|
|
- Configures ApiService to point to test server
|
|
- Shuts down server process after tests complete
|
|
|
|
## Debugging Tests
|
|
|
|
### Verbose Output
|
|
```bash
|
|
flutter test test/integration/dashboard_websocket_integration_test.dart --verbose
|
|
```
|
|
|
|
### Server Logs
|
|
- Server output is printed during test execution
|
|
- Look for "SERVER:" prefixed lines in test output
|
|
- Server errors are printed with "SERVER ERROR:" prefix
|
|
|
|
### Common Issues
|
|
|
|
1. **Port Conflicts**: Tests use random ports (8000-9000 range)
|
|
2. **Database Locks**: Ensure no other processes are using test databases
|
|
3. **Server Startup**: Tests wait up to 30 seconds for server startup
|
|
4. **Connection Timeouts**: WebSocket operations have 5-10 second timeouts
|
|
|
|
## Integration with CI/CD
|
|
|
|
The tests are designed to be CI/CD friendly:
|
|
- No external dependencies beyond Flutter/Dart and the xp_server
|
|
- Self-contained server process management
|
|
- Deterministic test data with golden databases
|
|
- Configurable timeouts
|
|
- Clean setup/teardown
|
|
|
|
### Example CI Usage
|
|
```yaml
|
|
- name: Run Dashboard WebSocket Integration Tests
|
|
run: |
|
|
cd xp_dashboard
|
|
./test/integration/run_dashboard_test.sh
|
|
```
|
|
|
|
## Extending Tests
|
|
|
|
### Adding New Test Cases
|
|
1. Add test case to `dashboard_websocket_integration_test.dart`
|
|
2. Follow existing pattern: setup → action → verify
|
|
3. Use `_waitForCondition()` helper for async state changes
|
|
4. Ensure proper cleanup in test teardown
|
|
|
|
### Adding New Golden Databases
|
|
1. Add scenario to `xp_server/test/create_golden_db.dart`
|
|
2. Implement data population function
|
|
3. Document scenario in this README
|
|
4. Test with `./run_dashboard_test.sh --db new_scenario.db`
|
|
|
|
### Testing New WebSocket Messages
|
|
1. Ensure server sends the message type during normal operation
|
|
2. Add listener in test to detect message reception
|
|
3. Verify dashboard state changes appropriately
|
|
4. Test error conditions and edge cases
|
|
|
|
## Benefits of This Approach
|
|
|
|
1. **True Integration**: Tests actual server-client communication
|
|
2. **No Fragile Mocks**: Uses real implementations, reducing test brittleness
|
|
3. **Realistic Scenarios**: Golden databases provide real-world data patterns
|
|
4. **Easy to Run**: Simple command-line interface with helpful options
|
|
5. **Comprehensive Coverage**: Tests WebSocket, HTTP API, and state management
|
|
6. **Debugging Friendly**: Server logs and verbose output for troubleshooting
|
|
|
|
## Performance Considerations
|
|
|
|
- Tests use random ports to avoid conflicts
|
|
- Server startup timeout prevents hanging tests
|
|
- WebSocket connections are properly cleaned up
|
|
- Database operations are optimized for test speed
|
|
- Memory usage is monitored for large datasets
|
|
|
|
## Security Notes
|
|
|
|
- Tests run on localhost only with random ports
|
|
- No external network access required
|
|
- Test databases contain no sensitive data
|
|
- All server processes are cleaned up after tests
|
|
- Temporary files are properly managed
|