How the same multi-family office classifies new securities without the model deciding
At an SEC-registered multi-family office, new securities landed in the master list every month without a type, a region or a risk class, and one person filled seven columns by hand. A script now applies the firm's own rules, fills the columns and marks what it cannot decide for a person. The rules live in the firm's folder and the firm edits them. Within eight days of the last session the compliance officer had rewritten the engine himself.
Seven columns, filled by hand
Every month the portfolio platform's master list gained securities nobody had classified. Asset type, region, currency, risk, liquidity and two more columns, looked up one by one and typed in by the operations person. The rules for deciding lived in one head and in a spreadsheet that had been rewritten many times. When that person was out, the columns waited.
The folder came first
Four folders before a line of code: input, output, audit log, reference. No regulation names AI. What an examiner rejects is a black box. So the firm has to be able to answer what went in, what happened and what came out, and the folder structure is that answer. It is the part of the build the compliance officer cared about most.
Code classifies. The model never does.
A script reads the export, applies the firm's rules and fills the seven columns. Where a rule does not reach, it writes 'review' and moves on. It does not guess. Every classification and every number comes from code; the model's job was to write and explain the script, and it never touches a row. Names never leave the firm: the operations person drops the client column from the export before anything runs, and the platform's own identifier survives, so a flagged row resolves back to a family inside the platform and no mapping file exists.
Rules the firm edits, with no release
The keyword lists, the corrections and the mapping tables sit in the firm's own folder and merge into the engine at runtime. A person who resolves a flagged security enters the decision once, and the next month the engine knows. An entry the engine cannot validate is reported and skipped, never accepted quietly. Only a structural change, a new asset class or a new region, needs a new version. That decision is what turned the engine from something I ran into something they own.
No output file is edited after a run
One run produced a file and a log that disagreed. The engine had flagged fourteen rows for review; afterwards the assistant layer, asked to help, resolved thirteen of them by hand in the output file while the log still said fourteen were waiting. The answers were right. That is what made it dangerous: a correct, unlogged edit inside a compliance officer's working file. The rule since then is written into the engine's instructions in plain words: you will often be able to work out the answer, and you still must not write it. A verify check now compares the log's review count with the file and flags any row carrying a manual tag. On that file it flags the mismatch and every edited row. On engine-produced files it returns clean.
He rewrote it in eight days
The stated win was never that the engine worked. It was that the compliance officer could build and change it himself. In the third session he wrote the firm's second engine with his own hands. Within eight days of that session he had rewritten this one, compiled it so it would run without a Python install, written a self-test and given it an audit log. Since then the firm has shipped its own releases. The engine, the rules and the log are theirs.
The flagged row
The decision on a security the rules cannot place. The engine says it cannot decide, a person decides, and the decision goes into the rules file with a name and a date. The firm could let the model take that call. It should not, and it does not.
Questions About This Case Study
No. Classification and every figure come from code that applies the firm's rules. The model wrote the script and explains it. A row the rules cannot place is marked for a person.
The engine reports what it could not read and stops on that part. It does not guess a column. A layout change is a structural change and gets a new version, tested against the previous month's file.
Yes. Keyword lists, corrections and mapping tables live in the firm's folder and merge at runtime. The firm has been shipping its own versions since the engagement ended.
If one person at your firm fills columns by hand every month from rules only they know, start there, not with a tool.
Start with a conversation. I’ll be honest about whether I can help.