Using Jupyter notebooks on BitXenial.es
Overview
The system allows authenticated users to launch personal Jupyter notebook instances with their own isolated environments and persistent storage.
Features
- User-specific Jupyter containers: Each user gets their own isolated Jupyter instance
- Persistent storage: User notebooks and data are preserved between sessions
- Authentication integration: Seamless integration with the existing auth system
- Automatic cleanup: Inactive instances are automatically cleaned up
- Pre-installed libraries: Data science stack (
pandas,numpy,scikit-learn,matplotlib, etc.)
Architecture
Components
-
Jupyter Manager (
astro/src/lib/jupyter-manager.ts)- Manages container lifecycle (create, start, stop, cleanup)
- Handles user-specific Docker volumes
- Tracks active instances
-
API Endpoints (
astro/src/pages/api/jupyter.ts)- POST
/api/jupyter- Start user’s Jupyter instance - GET
/api/jupyter- Check instance status - DELETE
/api/jupyter- Stop user’s instance
- POST
-
Proxy Service (
astro/src/pages/api/jupyter-proxy/[userId]/[...path].ts)- Proxies requests to user-specific Jupyter containers
- Handles authentication and session validation
-
User Interface (
astro/src/pages/jupyter.astro)- Web interface for managing Jupyter instances
- Start/stop controls
- Direct access to Jupyter Lab
Docker Integration
- Network: Uses
bitxenial_es_bitxenial_networkfor container communication - Ports: Dynamic allocation from range 9000-9099
- Volumes: User-specific persistent volumes for data and notebooks
- Image:
jupyter/datascience-notebook:x86_64-ubuntu-22.04
User Workflow
- Authentication: User signs in with credentials (admin/admin123)
- Access Protected Area: Navigate to
/protected - Launch Jupyter: Click on Jupyter Notebooks service card
- Start Instance: Click “Start Jupyter” to create container
- Access Notebook: Click “Open Jupyter” to launch Jupyter Lab
- Work: Create and run notebooks in the isolated environment
- Cleanup: Instances auto-stop after inactivity or manual stop
Container Management
User Containers
- Named:
jupyter_user_{userId} - Network: Connected to main application network
- Volumes:
jupyter_{userId}_dataandjupyter_{userId}_notebooks - Ports: Dynamically allocated and proxied through main app
Cleanup
- Manual cleanup script:
scripts/cleanup-jupyter.sh - Automatic cleanup: Configurable inactivity timeout
- Volume persistence: User data survives container restarts
Security
- Authentication: All requests require valid session tokens
- Isolation: Each user has separate container and volumes
- Network: Containers only accessible through authenticated proxy
- Token-based: Jupyter instances use unique tokens for additional security
Configuration
Environment Variables
BASE_PORT: Starting port for Jupyter containers (default: 9000)MAX_INACTIVE_MINUTES: Auto-cleanup threshold (default: 120)
Docker Network
- Ensure the main application has access to Docker socket
- Network name must match the deployed application network
Deployment
Prerequisites
- Docker daemon running on VPS
- Application container has access to Docker socket
- Port range 9000-9099 available for Jupyter instances
- Sufficient disk space for user volumes
Build Process
Include the new Jupyter files in the application build:
- Jupyter manager library
- API endpoints
- Proxy service
- UI components
Maintenance
Regular Tasks
- Run cleanup script periodically
- Monitor disk usage from user volumes
- Check for orphaned containers
- Review user access logs
Monitoring
- Container status via Docker API
- Volume usage tracking
- Session timeout monitoring
- Error logging for failed launches
Troubleshooting
Common Issues
Container fails to start
- Check Docker daemon status
- Verify network configuration
- Review port availability
- Check volume permissions
Proxy connection fails
- Verify container is running
- Check authentication tokens
- Review network connectivity
- Validate port mappings
Storage issues
- Check disk space on host
- Verify volume mount permissions
- Review cleanup configuration
Logs
- Application logs: Standard Astro application logs
- Container logs:
docker logs jupyter_user_{userId} - Cleanup logs:
/var/log/jupyter-cleanup.log
Future Enhancements
- Multi-language kernel support (R, Julia, Scala)
- Resource limits per user
- Shared workspaces for collaboration
- Integration with Git repositories
- Custom package management
- Advanced monitoring and metrics