unkinben
2309e9f43a
Five-service streaming platform: auth, catalogue, streaming, ingest, thumbnailer. Includes React frontend served by nginx, NATS JetStream event bus, aiobotocore async S3, PyAV video metadata + thumbnail extraction, service-to-service JWT auth, and a full unit + e2e test suite.
2.8 KiB
welcome to stream stack.
this project is to build a media streaming service comprised of a number of microservices and multiple frontends (desktop, mobile, admin). the aim is that every component is highly available and load balanced. state is shared between all processes through NATS, and persistent data is stored in pgsql or s3 (depending on the data). the backends should all be build using fastapi. each backend service should be able to run independently (should it be one pypi package that we enable features for different modules, or a different pypi package for each system?)
the frontend services should be in a fast and responsive language that will consume the fastapi services (react maybe?). there should be a "router" service that the frontend talks to, which proxies connections to the appropriate backend, or should be put different services on different dns addresses?
question: can we stream media from s3? will that enable skipping forward/backwards?
ensure there are unit tests for all file (in tests/) add a makefile that tests the unit tests (make test) using uvx (so we dont need to install any requirements permamently) add makefile test for linting with ruff add Dockerfile to run the streamstack (with booleans to enable different microservices) - this should use git.unkin.net/unkin/almalinux9-base:latest to build, then the uv container (dhi.io/uv:0.11) to run add docker-compose for e22 testing of the stack (with makefile targets to start/stop)
required projects:
- https://github.com/pyav-org/pyav
- https://github.com/fastapi/fastapi
- https://github.com/nats-io/nats.py
phase 1:
- build a backend microservice that can read media files with ffmpeg (pyav) from s3 and stream them. the url to stream the media should not include the name of the media. the url should be openable in mpv for testing.
phase 2:
- build a microservice that presents the media catalogue. this will be used by the frontend later to list media available.
phase 3:
- build auth microservice. it should be a jwt provider. when a user autheticates, they have a jwt kept somewhere that is passed to each microservice for each request. each microservice should then verify the jwt against the auth microservice.
phase 4:
- import microservice, for importing video into s3, adding to catalogue, finding metadata (thumbnail, actors, etc)
phase 5:
- simple react frontend (this is just for testing. no auth. just show catalogue and when you click on an item, play that video)
- the frontend should be its own container, so that it can be run in a DMZ
additional requirements: keep track of where a user is up to in a given video, so that when they replay it, it starts from a few seconds before where they stopped. when streaming video, send bursts of video to the user so that it caches on the client side