What exists today, and what goes wrong
Everything in this section comes from the background text of the problem statement, put into everyday words.
What is used today
Satellites, drones and ground stations mostly talk over radio today. The problem statement calls these "conventional radio-frequency systems". Radio works, but compared with light it is slow, it needs a licensed slice of the radio spectrum, and it can be upset by electrical interference.
FSOC (Free Space Optical Communication) sends the data on a laser beam instead. The problem statement lists its advantages: data rates from gigabits up to terabits per second, no spectrum licence needed, and high immunity to electromagnetic interference. That is why ISRO wants it for next-generation mobile networks.
Why it is hard
A radio signal spreads out in a wide cone, so a slightly wrong aim still works. A laser beam is, in the problem statement's words, "highly directional". It is a thin pencil of light. "Even a small angular error can prevent successful communication."
And the two ends are moving. Satellites orbit. UAVs fly and shake. So both terminals must keep finding and following each other while everything moves. That job is called PAT: Pointing, Acquisition and Tracking.
PAT has two stages. Coarse alignment gets the far terminal inside the camera's field of view. Fine alignment then locks the beam onto it precisely. This project is the coarse stage. The problem statement breaks it into four steps, and our software follows exactly those four:
- Observe the surrounding environment.
- Acquire and detect the remote terminal or beacon.
- Estimate its position.
- Continuously adjust the pointing direction to keep it in view.
The practical problem this project solves
The problem statement says it directly: "Developing and testing such algorithms on real hardware requires expensive cameras, pan-tilt mechanisms, and optical components and equipment." So it asks for "a software based virtual camera tracking" platform that is "inexpensive and accessible" for developing algorithms and for learning.
-
From the problem statement
The hardware is expensive
A real camera, a motorised pan and tilt mount and optical parts cost a lot and need a lab. That blocks students, small teams and anyone who wants to learn or try an idea quickly.
-
Our reasoning
Three more things a simulator fixes
You can switch on fog, rain or vibration at will instead of waiting for them. You know exactly where the beacon really is, so you can measure error precisely. And you can repeat the same test exactly, so you can tell whether a code change actually helped.
Words you will see on this page
- Beacon
- The bright spot of light from the far terminal. The thing we must find and follow.
- Screen
- The whole simulated scene, at least 2000 by 2000 pixels. The beacon moves around on it.
- Camera window
- The 640 by 480 pixel rectangle that shows where the laser terminal is pointing. It covers 4 by 3 degrees. The job is to keep the beacon inside it, near the middle.
- Pan and tilt
- Turning the camera window left and right, and up and down, at a limited speed of 5 to 10 degrees per second.
- Centre point
- The exact middle of the beacon spot, worked out to a fraction of a pixel. The rules call this the centroid.
- Lock
- The state where the software has found the beacon and is holding it inside the camera window.
What goes in, and what comes out
This is the part that is easiest to get wrong, so here it is as plainly as possible. The tracker looks at one thing, controls one thing, and produces one set of outputs.
What the tracker looks at, and what it controls
Looks at
- The whole scene picture, the screen. At least 2000 by 2000 pixels, black and white, 30 pictures a second.
- The picture already has the beacon in it, plus whatever disturbances are switched on: fog, shaking, noise.
- The problem statement calls this the "simulated video stream" and the "virtual camera feed".
- The tracker never gets told where the beacon really is. It only gets the picture.
Controls
- The camera window. A 640 by 480 rectangle that stands for where the laser terminal is pointing.
- It starts in the centre of the screen.
- It can move at most 5 to 10 degrees per second, like a real motorised mount. It cannot jump.
- The goal is to get the beacon inside this window fast, and keep it near the middle.
The three situations the software runs in
| Situation | What goes in | What the software does | What comes out |
|---|---|---|---|
| Your own run | Only your settings: screen size, camera window size and angle, beacon shape, size, start point and path, motor speed limits, which disturbances to add. No files needed. | Draws the scene, adds the disturbances, finds and follows the beacon, moves the camera window. | Live display. One log line per frame. A performance report at the end. |
| Benchmark 1 | Scenario files from the graders. A scenario is a set of settings for our simulator, not a video: which path, which disturbances, how strong. | Loads the settings and runs our own simulator with them, exactly like your own run. | The centroiding error log and the automatic performance report, which is what the graders mark. |
| Benchmark 2 | An .mp4 video from the graders at 30 frames per second. It shows the whole screen with noise and a moving beacon already in it. We only get these files at judging time, not while building. | Skips its own scene drawing completely. Each video frame is used as the scene picture. Finds and follows the beacon in it and moves the camera window over it. | The beacon centre point for every frame, compared by the graders with their reference values. Plus RMSE, time to first lock, time to re-lock after a loss, lock retention rate and FPS. This round tests our tracker, not our simulator. |
If the graders give us a video, why build our own scene at all
Because the video is one test at the very end. Generating the scene is the product itself.
-
Rule
It is a mandatory requirement
The "software shall be able to" list starts with "Generate a configurable virtual environment", "Generate one or more moving targets" and "Implement a movable virtual camera". Parameter rows 1 to 15 exist only to set up that generator. The 20 percent live demo is the graders watching it work.
-
Rule
Benchmark 1 cannot run without it
Benchmark 1 gives us scenarios, which are settings, not footage. Our own simulator has to run them, log the centroiding error and write the report. That is 30 percent of the marks that need the generator.
-
Practical
We will not have their video while building
Their .mp4 files arrive at judging. For the months before that, our own generator is the only source of test footage, in every weather and every motion pattern, as much as we want.
-
Practical
Only our own scene knows the true answer
To know whether the tracker is within 10 pixels, we must know where the beacon really is in each frame. In our own frames we know it exactly. That is how we measure error, catch mistakes when code changes, and train the AI with perfectly labelled pictures.
The loop, drawn simply
YOUR SETTINGS GRADERS' VIDEO (Benchmark 2) | | v | SCENE MAKER --> DISTURBANCES --> SCENE PICTURE <--------+ sky + beacon fog, shake, noise 2000 x 2000, 30 per second | v TRACKER find the beacon, measure its centre, predict where it goes next | v CONTROLLER which way to turn, how fast, never faster than the motor allows | +-------------------+-------------------+ v v CAMERA WINDOW MOVES LOG + REPORT 640 x 480, speed limited every frame, and a summary
What the report contains
The problem statement lists what the automatic performance report must have. We produce every item, with a one line definition printed next to each number so nobody has to guess what was measured.
| Item in the rules | What it means |
|---|---|
| Simulation duration | How long the run lasted. |
| FPS | How many pictures per second the software processed. Must be 20 or more. |
| Acquisition time | Time from the start until the beacon was first locked inside the camera window. Must be 2 seconds or less. |
| Average and maximum tracking error | How far the beacon sat from the middle of the camera window, on average and at worst. Must average 10 pixels or less. |
| Lock retention rate | The share of frames where the beacon stayed locked. Target loss must be under 5 percent. |
| Processing time | How long each frame took to process, on average and at worst. |
| Centroiding error | Asked for in both benchmark rounds. How far our measured centre point was from the true centre point, per frame, with its RMSE. |
| Re-acquisition time | Asked for in Benchmark 2. After a loss, how long until the beacon was locked again. Must be 1 second or less. |
Our solution, in plain words
The product has two halves. A simulator that stands in for the whole lab: scene, beacon, camera, motor and disturbances. And a tracker that finds and follows the beacon and steers the camera. Both run on an ordinary laptop with no special hardware.
What we are building
- A simulated sky. A 2000 by 2000 picture (or larger) that acts as the world, with stars or terrain in the background.
- A simulated beacon. A bright moving spot that can travel in a straight line, a circle, a figure of eight, at random, or in a spiral or wave. You can add more than one.
- A camera window. The 640 by 480 rectangle that shows where the laser terminal points. It starts in the middle.
- A simulated motor that moves the window. It obeys the same speed limit as a real motor, so the software cannot cheat by jumping.
- A disturbance generator. It dirties the picture the way the real world would: fog, rain, low light, air turbulence, platform movement, vibration and sensor noise.
- The tracker. It looks at each picture, finds the beacon, measures its centre exactly, predicts where it is heading, and tells the motor which way to turn.
- A scoreboard. It measures everything while the run goes on and writes the full report automatically when it ends.
What this changes
Before, with real hardware
- Needs a camera, a motor mount and optics. Expensive.
- Wait for fog or wind to test bad conditions.
- No way to know the beacon's true position, so accuracy cannot be measured.
- Every run is slightly different, so results cannot be compared.
- One lab, one team at a time.
After, with this software
- Runs on any laptop. Cost is zero.
- Switch fog on with a slider, any time.
- The simulator drew the beacon, so it knows the exact true position and can measure error precisely.
- Same settings and same seed give the exact same run, every time.
- Run hundreds of tests overnight, unattended.
Where AI comes in
AI is used at the exact point where the simple method breaks down, and nowhere else. Here is what each method does. Numbers marked as targets are what we are designing for, not results yet.
-
STEP 1 The simple method, which works most of the time
The beacon is a bright spot on a darker background. So look for bright pixels, group the ones that touch, and take their weighted average to get the centre. This is standard, well understood image processing.
It is very fast, and in clear conditions it can measure the centre to a fraction of a pixel. Target: under 3 milliseconds per frame on the full 2000 by 2000 picture.
Verdict: good enough on a clear day. We always run it first.
-
STEP 2 Where the simple method fails
In fog, rain or low light, the beacon becomes faint and the background brighter. "Find the brightest thing" now returns the wrong answer. Sensor noise makes fake bright specks, and salt and pepper noise at 10 percent of the picture makes thousands of them. If several bright objects are in view, brightness alone cannot say which one is ours.
Verdict: exactly the conditions the rules ask us to survive.
-
STEP 3 The AI model takes over
A small neural network looks at the picture and produces a heat map: how likely each spot is to be the real beacon. It has learned what a beacon looks like, its shape, its soft edges, how it sits in noise. So it still finds the spot when the spot is faint.
The model is kept deliberately small so it runs on a normal laptop processor with no graphics card. Once the beacon is locked, the model only looks at the area around where the beacon is expected, which keeps it fast. Target: under 5 milliseconds per frame.
Verdict: this is the part that rescues the hard cases.
-
STEP 4 Where the training data comes from
Training an AI normally means collecting thousands of pictures and paying people to label each one by hand. We do not have to. Our own simulator draws the pictures, so it already knows exactly where the beacon is in each one.
That gives unlimited, perfectly labelled training pictures for free, in every weather we can generate. The simulator is both the test bench and the data factory.
Verdict: no dataset to buy, no hand labelling, no labelling mistakes.
-
STEP 5 Knowing which target is yours
The rules allow more than one beacon. The software must keep following the one it was told to follow, even when two cross paths. It does this by combining how each target looks with how each has been moving. The one that matches the expected path is ours.
Verdict: identity is held through crossings, not guessed again each frame.
-
STEP 6 Predicting the next position
The tracker keeps a model of how the beacon is moving. It runs several motion models at once, moving straight, speeding up, and turning, and trusts whichever one is matching reality. This covers every path type in the rules without retuning.
So if the beacon vanishes into noise for a moment, the camera window does not freeze or wander. It keeps moving to where the beacon should be, and picks it up again quickly. This is how we aim to meet the 1 second re-acquisition limit.
Verdict: prediction turns a brief loss into a non event.
The numbers behind the design
None of these appear in the rules directly. They come from simple arithmetic on the numbers that do. One assumption is needed, and it is stated first.
- One pixel equals
- 22.5arcsec 4 degrees divided by 640 pixels
- Allowed error
- 10pixels that is 0.0625 degrees
- Window can move
- 26.7px/frame 5 degrees per second, at 30 frames per second
- Platform can drift
- 20.0px/frame at its maximum setting, and vibration adds up to 20 more
- Windows per screen
- 13areas a 2000 by 2000 screen holds about 13 camera windows
At the maximum platform motion setting, three quarters of the motor's speed goes on cancelling the drift, and only a quarter is left for actually following the beacon. This is why the controller must predict, not just react.
What this means for finding the beacon
Because the tracker looks at the whole screen, spotting the beacon takes only a few frames. Turning the window from the centre to the farthest corner then takes about 1 second at 5 degrees per second, with both axes moving together. Add a few frames to confirm the detection is real and the total sits inside the 2 second limit from any starting point.
There is also a hard mode setting, where the tracker is allowed to see only the inside of the camera window, as a real narrow camera would. Then it has to sweep the screen in a spiral. The screen holds about 13 window sized areas, so a beacon hidden in a far corner takes several seconds to reach, and the log records that honestly. This mode is for testing the algorithm under harder conditions, not the default.
-
Design rule 1
The window cannot jump
It is tempting to just place the window on top of the beacon every frame. That is not allowed. The motor has a speed limit, and the drift is eating most of it. So the controller works out a turn rate every frame, never above the limit, and uses the predicted position to lead the target rather than chase it.
-
Design rule 2
Log both errors, with clear names
The rules talk about tracking error in the specifications and centroiding error in the benchmark rounds. We measure both every frame. Tracking error: how far the true beacon sits from the middle of the window. Centroiding error: how far our measured centre is from the true centre. Whichever a grader means, the number is there with its definition.
-
Design rule 3
Do not chase the vibration
Vibration jerks the picture by up to 20 pixels every frame in a random direction. A motor cannot follow that and should not try. The tracker smooths it out and the controller follows the smoothed position. It also measures how much the whole picture shifted between frames and cancels that drift directly.
How the software is built
Six layers. Three of them do the real work and pass information round in a loop: the World draws the picture, Perception finds the beacon, and Control moves the window.
The layers
DISPLAY What you see: the screen with the window drawn on it, live graphs, settings ^ SCOREBOARD Collects every measurement, writes the CSV log and the PDF report ^ CLOCK Runs everything at a steady 30 pictures per second, same seed = same run | | | WORLD PERCEPTION CONTROL the sky where is the beacon which way to turn the beacon(s) simple detector decide the state the camera window AI detector work out the error the motor exact centre point add the prediction disturbances which one is ours respect the speed limit draw the picture predict where it goes send the command ^______________ turn command goes back to the motor ______|
What happens in one frame, thirty times a second
- The beacon moves a little. The platform drifts and shakes a little.
- The motor moves the camera window by however much it was told last frame.
- The software draws the scene picture, then dirties it with fog, turbulence, drift, shake and sensor noise. In Benchmark 2 this step is skipped and a video frame is used instead.
- The picture goes to the simple detector. If the detector is unsure, the AI model is asked as well.
- The exact centre of the beacon is measured, to a fraction of a pixel.
- The predictor updates its idea of where the beacon is going.
- The software decides its state: searching, checking, following, coasting, or re-finding.
- It works out how far the beacon is from the middle of the window and turns that into a motor command, capped at the motor's speed limit.
- Every number from this frame is written to the log and drawn on the live graphs.
The five states the tracker moves through
| State | What it means | What the window does |
|---|---|---|
| SEARCH | No beacon found yet | Looks at the whole picture for it. In hard mode, sweeps outward in a spiral. |
| VERIFY | Found something, but is it real | Waits for 3 good frames out of 4 before trusting it, so a noise speck is not mistaken for the beacon |
| TRACK | Locked on and following | Moves so the beacon stays in the middle |
| COAST | Lost sight of it for a moment | Keeps moving to where the prediction says it should be |
| REACQUIRE | Still missing, go and find it | Looks again around the predicted spot, widening as the uncertainty grows |
How the four steps in the rules map onto the software
| Step in the rules | Our software |
|---|---|
| Observe the environment | The tracker receives the full scene picture every frame |
| Acquire and detect | Simple detector, then AI detector when needed, then the VERIFY check |
| Estimate the position | Exact centre measurement plus the motion predictor |
| Continuously adjust pointing | The controller moves the camera window every frame within the speed limit |
What each part does
| Part | Its job, in plain words |
|---|---|
| Scene maker | Draws the background: a star field, ground terrain, or plain sky. You set the screen size, the window size and its angle of view. Can add other bright objects to test the tracker on purpose. |
| Beacon maker | Creates one or more moving spots. You pick the shape, the size from 5 to 20 pixels, the start point and the path: straight, circular, figure of eight, random, spiral or wave. |
| Window and motor | Keeps track of where the camera window is pointing. The motor has a top speed, a top acceleration and a small delay, like a real one. |
| Disturbance generator | Adds fog, rain, haze and low light, plus air turbulence, platform drift, vibration and sensor noise. Each one can be switched on or off on its own and set to any strength up to the limits in the rules. |
| Detector | Finds the beacon in the picture. Runs the fast simple method first, then the AI model when the picture is hard. Tells several targets apart. |
| Predictor | Keeps track of how the beacon is moving and estimates where it will be next. Runs several motion models at once and trusts whichever is matching reality. |
| Controller | Turns "the beacon is 40 pixels left of the middle" into "move left at 3 degrees per second", never over the speed limit, never over correcting, and leading the target using the prediction. |
| Scoreboard | Records about 30 numbers every frame, then writes a CSV file and a PDF report with graphs, pass or fail against each rule, and the settings used. |
How the picture gets dirtied, in the right order
In nature these things happen in a particular order. We copy that order instead of just adding noise at the end.
| Order | What happens | Rule it comes from |
|---|---|---|
| 1 | Fog, rain or haze absorbs light and washes out contrast and brightness | Atmospheric disturbance: Clear, Haze, Fog, Rain, Low light |
| 2 | Moving air bends the light, so the spot wanders about | Atmospheric turbulence |
| 3 | The same moving air makes the spot flicker in brightness | Atmospheric turbulence |
| 4 | The spot blurs and spreads out | Atmospheric turbulence |
| 5 | The whole platform drifts, up to 20 pixels per frame, in a straight line or another pattern | Platform motion |
| 6 | The platform vibrates quickly, up to another 20 pixels per frame | Camera jitter |
| 7 | The camera sensor adds grain and dead pixels | Image noise: salt and pepper, Gaussian, Poisson |
How we build it, step by step
Something working exists from step M2. After that, every step adds one measurable improvement, and the numbers go straight into the technical report.
| Step | What we build | How we know it is done |
|---|---|---|
| M0 | Project skeleton, settings file, logging, video file input | It runs, reads an .mp4, and writes a log |
| M1 | The scene, the beacon, the camera window, the motor | Produces 30 pictures a second with the true answer attached |
| M2 | Simple detector, centre measurement, basic motor control | Follows a straight moving beacon within 10 pixels |
| M3 | All five states, timing measurements | Locks in under 2 seconds, re-locks in under 1 |
| M4 | All the disturbances | Every effect in the rules can be switched on and set |
| M5 | Prediction, drift cancelling, smarter control | Keeps lock more than 95 percent of the time at maximum drift |
| M6 | Train the AI, plug it in, handle several targets | Works in fog and low light where the simple method alone fails |
| M7 | Automatic report, batch testing | Every run ends with a PDF; 50 runs complete overnight |
| M8 | The full screen interface and the installer | Runs above 20 frames a second on a fresh laptop |
| M9 | Report, user manual, demo video | Everything on the deliverables list is finished |
Checklist against the rules
Every requirement in the problem statement, and where a grader can actually see it working.
The eight things the software must do
| Requirement, as written | Where a grader sees it |
|---|---|
| Generate a configurable virtual environment | The settings panel and the scenario file |
| Generate one or more moving targets | The target list and the trails on the display |
| Implement a movable virtual camera | The camera window moving across the screen |
| Detect the target beacon automatically | The marker drawn on the detected beacon |
| Track the beacon continuously using computer vision | The centre marker and the prediction marker |
| Control and reposition the virtual camera | The motor command graph and the state banner |
| Generate and introduce disturbances in the virtual camera feed | The individual switches in the disturbance panel |
| Display tracking performance and statistics in real time | The live graphs and the status bar |
The full parameter list, rows 1 to 25
| Row | What the rules ask for | What we do |
|---|---|---|
| Camera parameters | ||
| 1 | Screen size at least 2000 x 2000 pixels, user defined | Adjustable, starts at 2000 x 2000 |
| 2 | Monochrome focal plane array camera, colour optional | Both, with a switch |
| 3 | Camera resolution 640 x 480, user defined | Adjustable |
| 4 | Camera FOV user defined, default 4 by 3 degrees | Adjustable; the degrees per pixel is worked out from it |
| 5 | Camera update rate at least 30 Hz | Steady clock, adjustable from 30 upward |
| 6 | Initial camera position at the centre of the screen | Yes, by default |
| Target parameters | ||
| 7 | Target type: beacon spot | A soft glowing spot, drawn realistically |
| 8 | One target mandatory, multiple optional | As many as you like, with one marked as the one to follow |
| 9 | Target shape user defined, default square | Square, circle, soft spot or custom image |
| 10 | Target size 5 to 20 pixels, default 10 x 10 | Adjustable |
| 11 | Initial target location user defined, default random | Random, or fixed if you prefer |
| 12 | Motion: straight line, circular, figure of 8, random; optional spiral, sinusoidal, user defined | All of them, plus custom paths |
| Camera motion constraints | ||
| 13 | Max pan speed 5 to 10 degrees per second, default 5 | Enforced by the speed limiter |
| 14 | Max tilt speed 5 to 10 degrees per second, default 5 | Enforced by the speed limiter |
| 15 | Update interval at least 20 Hz | Commands go out every frame at 30 Hz |
| Performance specifications | ||
| 16 | Acquisition time 2 seconds or less | measured pass or fail printed in the report |
| 17 | Tracking error 10 pixels or less | measured pass or fail printed in the report |
| 18 | Target loss under 5 percent | measured pass or fail printed in the report |
| 19 | Re-acquisition time 1 second or less | measured pass or fail printed in the report |
| 20 | Processing speed at least 20 FPS | measured pass or fail printed in the report |
| Disturbances and noise | ||
| 21 | Image noise: salt and pepper around 10 percent, Gaussian, Poisson; one or more | All three, any combination |
| 22 | Max standard deviation of noise 20 pixels, user defined | Adjustable |
| 23 | Max camera jitter 20 pixels per frame either way | Adjustable up to the limit |
| 24 | Atmospheric disturbance: clear, haze, fog, rain, low light; user defined reduction in contrast and brightness | Five presets, plus manual contrast and brightness control |
| 25 | Platform motion up to 20 pixels per frame; linear mandatory; circular, random, spiral, figure of 8 optional | All of them |
What we hand in
| Deliverable, as written | How it is produced |
|---|---|
| Software application: a standalone executable | A single installer for Windows and Linux, no setup needed |
| Source code: modular, documented, commented | Separate modules with clear boundaries and documentation |
| Technical report, 10 to 15 pages: problem understanding, architecture, modules, tracking methods, AI methods, test methodology, performance analysis, future improvements | Written from the real measured numbers, section for section |
| User manual: installation, operation, parameter configuration, GUI description | Written alongside the interface |
| Optional 3 to 5 minute demo video | Recorded from inside the application |
| Performance log: duration, FPS, acquisition time, average and maximum tracking error, lock retention rate, processing time | Written automatically at the end of every run |
Where the marks actually are
The rules give the marking scheme. Sixty percent sits in the two benchmark rounds, where graders hand us their own files. Those rounds are won by being ready, not by being clever.