Momento project includes 5 parts:
React frontend app for Operators (./client-operator)
React frontend app for Fans (./client)
C++ app for cropping big images into smaller ones based on provided coordinates and size (./momento-cropper)
C++ Test uploader to emulate work of NUC for testing purposes (./nuc-uploader)
Node.js backend for running frontend apps and business logic written with Express.js, С++ app for cropping
Frontend
Both React apps have a similar stack:
MobX for data management. All data is contained separated from the React components into domain-based MobX services that load data via Axios;
Ant Design + Styled-components for UI;
Date-fns for date manipulations;
Operator’s frontend
uses socket.io-client for fetching updated data regarding newly created moments of the game; updated moments
all images from the app are being uploaded via FormData as File types;
stadium seat mapping is implemented using Canvas. Mapping logic includes 3 services: mapping.ts, canvas.ts, mouse.ts.
canvas.ts service tracks user actions and draws marks on the HTML canvas, notifies mapping service about mapping updates
mapping.ts service tracks mapped seats, manages sections (update, creation, deletion). Every time changes are made, the operator can manually save them.
Fan’s frontend
all data that needs to be stored for multiple sessions (selected seat, game, edited moments) is being stored via service-worker. It’s done to achieve the same functionality across all platforms (including installed PWA apps on IOS)
received moments and memento can be edited by fans using Canvas. All data is being stored and automatically
All moments and momento are being displayed inside Canvas, previously cropped according to edits considering width, height, x, y, and aspect ratio.
Downloading and sharing done with using default Web APIs
Node.js Backend
serves fan’s frontend app as ‘./’ and operator’s frontend app as ‘./operator’
saves all data from the frontend apps to MongoDB.
Uses AWS S3 bucket to store momentos
Main technical features
All momentos are being uploaded to AWS via nuc-uploader app written on C++(uploader.exe)
Steps to configure the NUC and bind it to the camera are described below.
To add the camera go to the operator’s portal (OP) and then
1. Create the stadium or choose an existing one
2. Add the camera to this stadium
To set up the NUC which is connected to the camera you need
1. Install aws-cli (execute command in the terminalmsiexec.exe /i https://awscli.amazonaws.com/AWSCLIV2.msi)
2. Login there
$ aws configure
AWS Access Key ID [None]: AKIAIOSFODNN7EXAMPLE
AWS Secret Access Key [None]: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
Default region name [None]: us-west-2
Default output format [None]: json
3. Place uploader.exe on the NUC, in any directory
Then run uploader.exe via powershell from the directory where uploader.exe is located with the following command
.\uploader.exe s3:///live_moments///
In practice, the command looks like this
.\uploader.exe C:\testfolder\ s3://mvp-test-camera1/live_moments/Stamford_Bridge/250_backyard/
Stamford_Bridge – stadium name created on the OP
250_backyard – camera name created on the OP which is bind to this stadium
Slashes (\ and /) MUST be put on the end in both cases
There MUST NOT be any spaces in this path (e.g. if the stadium name is Stamford Bridge use Stamford_Bridge instead)
s3:///live_moments///
So, by running this command, we pass the name of the camera and the name of the stadium to the NUC, and then, under the hood of the script, all the files placed in the directory C:\testfolder\ (directory we passed to uploader.exe to watch file uploads) will be uploaded to the AWS S3 according to the cloud upload instructions I created before
after all these steps it is possible to put some photos from the camera to the folder, they will be put to the AWS S3 bucket and the momento backend will be notified about it and all the needed momento management stuff will be triggered
2. Camera Moment creation logic
When each momento is uploaded to the AWS S3 bucket, AWS Lamdba triggers the endpoint (service for this endpoint is located in the /root/server/src/momento-listener.ts file) which is responsible for joining momentos into moments. Two momentos are considered to be from the one moment when the difference between two momentos timecodes should be less than or equal to a certain value. This logic is based on values set in the config for each sport type.
3. Stadium moment creation logic
After a moment from each camera is created, the backend groups them in stadium moment (general moment). If there is no stadium moment (there are no general moments in the database first moment of which has timecode difference with input moment greater or less than values in the config for each sport type described above) it is being created else stadium moment is being updated in the database and event emitter triggers GENERAL_MOMENT_CREATED/GENERAL_MOMENT_UPDATED event which triggers socket.io server. Then socket.io server emits similar actions to the operator’s portal so the operator can be notified about the new moment which has to be reviewed.
4. Momento cropping and caching
Each time momento is being uploaded to the AWS it is cached. Initial uncropped momentos are kept in the cache until the total cache amount is under the predefined value (100 GB for now, can be altered in the /root/server/src/moment-cropper.ts file) then the oldest ones will be overridden by new momentos. Cropped momentos are stored in the cache for 1 hour from the latest fan viewing (/root/server/src/cache-s3.ts is responsible for caching).
All momentos are cropped with C++ application (/root/momento-cropper) which is triggered by /root/server/src/moment-cropper.ts service. When the initial momento is uploaded to the AWS the app converts it to raw view and keeps it in the cache. Then when a fan asks to view his personal momentos, the app crops images in the raw view, and then they are converted to the .jpg format via ffmpeg. Also, uncropped momentos are being compressed and stored in the cache so the operator portal can process them much faster than initial big images so the operator can look through them briefly.
PWA: https://mymomento.com
