This project is a Scrum exercise.
The market data is where you practise it.
The market data is where you practise it.
You are not training to be traders or financial analysts. You are learning to run a professional delivery cadence — planning, execution, review, retrospective — with AI tools doing the repeatable work and humans providing the judgment. That combination is what the workforce is hiring for right now.
Why Scrum and AI Belong Together
Scrum was designed for problems where the requirements aren't fully known upfront — where you learn by doing, deliver in short cycles, and improve based on what you find. That is exactly the environment AI tools operate in today.
AI handles the repeatable execution — fetching data, running analysis, calling models, logging outputs. Scrum handles the continuous improvement — sprint goals, daily cadence, retrospectives, Definition of Done. Neither is sufficient without the other.
The compounding effect
A team that uses AI without Scrum produces good outputs inconsistently. A team that uses Scrum without AI works hard but slowly. A team that uses both — with a disciplined retrospective each sprint — builds a compounding advantage. Each sprint the automation gets slightly better, the process gets slightly tighter, and the output gets slightly more accurate. Over eight sprints, that compounds into something significant.
The key insight from this project
By Sprint 6, the best teams have a GitHub Actions workflow that fires automatically after the US market closes on Friday (Saturday morning SGT), collects all instrument data, runs agent scripts, calls LLMs via API, and commits the output — without anyone touching a keyboard. That didn't happen because they used smarter AI. It happened because every retrospective produced one concrete improvement to the automation. Scrum compounded what AI started.
10 Scrum Principles — Applied to This Project
These are the principles your team should be improving on every sprint. Each one has a project-specific meaning.
01
A sprint goal is a commitment, not a description
Weak: "We will work on the pipeline." Strong: "We will deliver a fully automated end-to-end pipeline that runs without manual intervention and is sealed with vW28 by Sunday midnight." If you can't tell at midnight Sunday whether you met it — it's not a goal.
R1 writes this. R2 holds the team to it.
02
Done means done. Not mostly done.
A pipeline task that is "almost finished" is not a sprint increment. The Definition of Done is a checklist — if an item isn't checked, the sprint isn't done. Tasks that linger across sprints blur your velocity and make it impossible to measure progress.
The DoD is the contract your team makes with itself.
03
Agree on timeboxes early — enforce them even if they feel wrong
Define timeboxes for every Scrum event: sprint (one week), daily standup (15 min), sprint review (per-team slot), retrospective (30 min). An incorrect timebox you enforce teaches you more than no timebox at all. Adjust by consensus in the next retrospective — never ad hoc.
R2 owns the clock.
04
A blocker surfaced Wednesday can be resolved. Sunday cannot.
The daily standup exists precisely for this. If your pipeline broke on Thursday and you didn't say anything until Sunday — that's not a pipeline problem, it's a communication problem. R2 must make it psychologically safe to raise blockers early.
Hiding a blocker is not Scrum.
05
The daily standup is the heartbeat of Scrum
15 minutes. Stand up. Three questions: what did I do, what will I do, what is blocking me. One person speaks at a time. No laptops. Defer detailed discussions until after. If it's running 30 minutes, that's a symptom — someone is problem-solving instead of flagging.
Short, sharp, every day.
06
Velocity is an outcome — you can't optimise for it without gaming it
Never add backlog items to hit a velocity target. Never inflate story points to look productive. Velocity is a planning tool, not a performance metric. Your sprint goal drives what goes in — velocity follows from honest work.
Focus on the goal. Velocity follows.
07
The demo either runs or it doesn't
Come to Sprint Review prepared. Never create the demo on the spot. Show the GitHub Actions run log — not a screenshot of code that "works on my machine." Incomplete items go back to the backlog, not into the demo as "nearly done." Well-structured demos build stakeholder confidence.
If it doesn't run in the room, it doesn't count.
08
The retrospective is where the team gets better — not the pipeline
Retrospective is separate from Sprint Review. Every team member speaks — not just the loudest voices. The goal is to identify problems and plan when to address them — not solve everything live. One concrete change committed for the next sprint. Written in the repo.
Small consistent improvements turn Scrum from process into culture.
09
Mid-sprint interruptions are a tax on velocity
When something urgent arrives mid-sprint, log it in the backlog and address it next sprint — unless it is truly blocking delivery. Don't estimate the new work; just know it cost you something. If your team is regularly interrupted, surface that pattern in the retrospective and decide how to protect the sprint goal.
Protect the sprint goal. Everything else is backlog.
10
Sustainable pace — the sprint that breaks you helps no one
Set realistic sprint goals. Distribute intensive tasks across team members. If your team is consistently finishing at 11:55pm Sunday, the goal was too ambitious or the work wasn't distributed. Scrum is designed to be sustainable across eight sprints — not heroic for one.
The team that lasts eight sprints beats the one that burns out at four.
Scrum Events — What They Mean in This Project
| Scrum Event | In This Project | AI Angle |
|---|---|---|
| Sprint Planning Monday · in class |
R1 writes sprint goal. R2 assigns roles. DoD checklist written. DECISION.md committed. | The sprint goal defines what the AI pipeline must produce by Sunday. Writing it forces clarity about what "automated" actually means for this team this week. |
| Daily Scrum 15 min · every day |
Three questions. Blockers surfaced immediately. R2 chases by Wednesday. | Pipeline failures, API key issues, GitHub Actions errors — these are Wednesday blockers, not Sunday surprises. The daily standup is what catches them early. |
| Sprint Execution Mon–Sat |
Agents commit independently (R3–R6). LLM API calls run (R8). Automated fetch fires Saturday morning SGT after Friday US market close (R9). | The best teams have AI doing the execution. Humans do the judgment — Wild Card (R7), calibration (R10), pipeline monitoring (R9). The sprint cadence keeps this ordered. |
| Sprint Review Monday presentation |
Every role has a speaking point. Show GitHub Actions log live. Demo the pipeline running — or explain the failure honestly. | The Sprint Review is where the AI system is evaluated against reality. Did the automated fetch run? Did the LLM comparison table generate correctly? Did the prediction reflect what the agents produced? |
| Retrospective After Sprint Review |
Separate from Sprint Review. Every member speaks. One concrete change for next sprint. Committed to repo. | The retrospective question for an AI team is specific: did the automation run without failure? Did the output improve? What checklist item prevents the failure next sprint? These are engineering questions — and they compound. |
Your Role · Scrum · Workforce
Every role in this project practises a real skill. The Scrum framework is what makes those individual skills add up to a functioning system.
R1
Product Owner
Scrum: owns the sprint goal and DoD
→ Product Manager in industry
R2
Scrum Master
Scrum: protects cadence, removes blockers
→ Team Lead / Agile Coach
R3
Almanac Agent
Scrum: delivers a sprint backlog item (historical analysis) on time
→ Research Analyst
R4
Macro Agent
Scrum: delivers a sprint backlog item (macro analysis) on time
→ Macro / Strategy Analyst
R5
Technical Agent
Scrum: delivers a sprint backlog item (technical analysis) on time
→ Data Analyst
R6
Data Collector
Scrum: owns the infrastructure that feeds the sprint
→ Data Engineer
R7
Human Score Lead
Scrum: the Product Owner's counterpart — applies human acceptance criteria that AI cannot
→ Senior Analyst / Decision Maker
R8
LLM Operator
Scrum: builds the AI synthesis layer — the highest-value technical backlog item
→ AI / ML Engineer
R9
GitHub Lead
Scrum: owns the Definition of Done — nothing is done until the tag is cut
→ DevOps / Platform Engineer
R10
Calibration Lead
Scrum: generates the retrospective data — did the system improve?
→ Product Analyst / KPI Owner
05
The daily standup is the heartbeat of Scrum
15 minutes. Stand up. Three questions: what did I do, what will I do, what is blocking me. One person speaks at a time. No laptops. Defer detailed discussions until after. If it's running 30 minutes, that's a symptom — someone is problem-solving instead of flagging.
Short, sharp, every day.
11
The Scrum Master's value is proportional to how early they surface what is not working
A dependency that surfaces on Sunday was visible on Wednesday. The SM who discovers a blocker on Sunday night did not fail on Sunday — they failed on Tuesday when they did not check in. The Wednesday Test: by mid-week, R2 must know the status of every role individually. Not from a group chat. Individually. One message, one person, one status.
Wednesday discovery = solvable. Sunday discovery = damage control.
12
Dependency collapse is a Scrum Master failure — not a team member failure
When R8 cannot synthesise because R3 did not commit, the team blames R3. The correct diagnosis is that R2 did not know R3 was behind until it was too late to fix it. R2's job is not to do R3's work. It is to know by Wednesday whether R3's work will be done before Thursday — when R8 needs it. The pipeline has a strict execution order. R2 watches the chain, not the deadline.
The SM does not wait for the standup to find out what people are doing.
13
The mid-week check-in is not a group message — it is individual, documented, and acted upon
Posting "hey how is everyone going?" in the team channel is not a check-in. R2 contacts each role member individually: "R3 — your Almanac output needs to be committed by Wednesday noon. R8 is blocked until it lands. What's your status?" The response — and R2's action if the response reveals a problem — is committed to the repository as a standup note. If it is not documented, it did not happen.
A check-in that produces no documentation helped no one.
10 Lines Worth Saying Out Loud Every Sprint
One per class session. These are the ideas that compound.
"A sprint goal is a commitment, not a description." — If you can't tell at midnight Sunday whether you met it, rewrite it.
"Done means done. Not mostly done." — A task that is almost finished is not a sprint increment. It goes back to the backlog.
"Velocity is an outcome. You can't optimise for it without gaming it." — Focus on the goal. Velocity follows from honest work.
"A blocker surfaced Wednesday can be resolved. Sunday cannot." — The daily standup exists precisely for this.
"The retrospective is where the team gets better — not the pipeline." — One concrete change per sprint compounds over eight sprints.
"Build the pipe. Don't carry the bucket." — Calling an LLM via API and pasting from a chat window both produce output. Only one scales.
"Small consistent improvements turn Scrum from process into culture." — Sprint 8 belongs to the team that improved every sprint, not the one that sprinted hardest once.
"The demo either runs or it doesn't. There is no 'we were close'." — If it doesn't run in the room, it goes back to the backlog.
"The data domain is the training environment. The professional habits are what transfer." — You are not learning to predict markets. You are learning to deliver a working system on a cadence.
"If it's not on the DoD checklist, it won't get done." — The Definition of Done is not paperwork. It is the only thing standing between a completed sprint and a forgotten one.