Evidence before complexity
The simpler approach performed better.
A reproducible comparison on 24 hand-authored synthetic development cases: 16 single-team requests and 8 intended for human review. These results describe this small test set, not production performance.
Current development-set results · 32 synthetic training examples| Approach | Expected decisions matched | Auto-suggestion coverage | Accuracy among suggestions | Review cases caught |
|---|
| Keyword rules | 23 / 24 (95.8%) | 70.8% | 94.1% | 7 / 8 |
|---|
| TF-IDF matching | 22 / 24 (91.7%) | 75.0% | 88.9% | 6 / 8 |
|---|
Recommendation: keep the rule baseline as the default prototype.TF-IDF suggests an owner more often, but makes more mistakes. Keep staff confirmation for both. No claim of time savings, real-world accuracy, or readiness for unattended routing is supported.
Inspect every test case and failure
| Request | Expected | Rules | TF-IDF |
|---|
| The sink keeps leaking | Facilities | Facilities | Facilities |
| Please repair the broken window latch | Facilities | Facilities | Facilities |
| The shower drain needs repair | Facilities | Facilities | Facilities |
| My heater needs replacement | Facilities | Facilities | Facilities |
| I need a password reset | IT Support | IT Support | IT Support |
| My laptop cannot connect to wifi | IT Support | IT Support | IT Support |
| The portal login is locked | IT Support | IT Support | IT Support |
| My ethernet connection keeps dropping | IT Support | IT Support | IT Support |
| Please cancel my housing agreement | Housing Administration | Housing Administration | Housing Administration |
| I have a question about the move out deadline | Housing Administration | Housing Administration | Housing Administration |
| I need a room transfer | Housing Administration | Housing Administration | Housing Administration |
| My housing bill was charged twice | Housing Administration | Housing Administration | Housing Administration |
| I need help with a roommate conflict | Residence Life | Residence Life | Residence Life |
| My neighbors are making noise during quiet hours | Residence Life | Residence Life | Residence Life |
| How can I join community social activities | Residence Life | Residence Life | Residence Life |
| I want to meet the resident assistant | Residence Life | Residence Life | Residence Life |
| What should I do | Human review | Human review | Human review |
| Can you help me tomorrow | Human review | Human review | Human review |
| I need to discuss something | Human review | Human review | Human review |
| The sink is leaking and wifi keeps dropping | Human review | Facilities | Human review |
| My roommate is noisy and I need a password reset | Human review | Human review | IT Support |
| The heater is broken and my housing bill is incorrect | Human review | Human review | Facilities |
| I lost a blue umbrella | Human review | Human review | Human review |
| I need a dining meal plan refund | Human review | Human review | Human review |
The evaluation set informed development: a minimum-evidence guard was added after a regression test. This is not an untouched holdout. Paraphrases, negation, new queues, and sensitive scenarios need independent evaluation.