LLM Inference & Evaluation
I build and maintain Biexce’s internal platform for coding models. My work covers model serving, benchmark design, and agent workflows that connect to developer tools.
Coding model evaluation
Use the same coding tasks and project context for each configuration so differences in output are easier to interpret.
Serve each model through vLLM. Measure when each response starts and ends, along with aggregate throughput and GPU memory use.
Compare the generated changes with the task requirements. Review serving metrics and failed runs alongside code quality.
- Role
- AI Engineer · Biexce
- Scope
- Internal developer platform
- Stack
- Docker · vLLM · Bifrost · MCP
Serving models on GPUs
Developer tools call a shared API through Bifrost. The gateway routes requests to vLLM, which serves the model in a Dockerized runtime on GPU servers. I monitor the services and troubleshoot model and infrastructure failures.
Multi-agent coding workflow
I build multi-agent workflows for planning, implementation, testing, and code review. MCP connects the workflow to Unity, Git, and internal documentation. The planner turns a task into acceptance checks, and the workflow carries the proposed diff and test report into review.
Illustrative Unity task
Wait for the last enemy before advancing the wave
Define what the fix must do
The planner reads the task and the project’s wave rules. It identifies the wave controller and match reset as the parts to inspect, then turns the bug report into acceptance checks.
Acceptance checks
- Keep the current wave active while enemies remain.
- Advance once after spawning finishes and the last enemy is gone.
- Clear the pending transition when the match restarts.
Change the wave transition
The coding agent uses Unity project context and the Git working tree to locate the existing transition logic. It makes a scoped C# change and hands the diff to the test stage.
Proposed Git changes
- Gate the transition on completed spawning and zero active enemies.
- Guard against duplicate advance requests.
- Cancel the pending transition in the match reset path.
Exercise the changed behavior
The test stage checks that the project compiles, then uses Unity Play Mode tests to exercise the wave lifecycle. If a check fails, the expected and observed behavior go back to implementation.
- Spawning continues · wave stays activePass
- Enemies remain · wave stays activePass
- Final enemy removed · wave advances oncePass
- Restart · pending transition clearsPassFail
The previous wave’s transition still runs after restart. Send that reproduction back with the failed check.
Back to implementationReview the diff and test results
The review agent checks the patch against the original task, inspects the test report, and flags changes outside the requested scope. The developer gets the diff and the evidence behind the recommendation.
Review decision
Ready for developer review
The wave checks passed. Send the scoped diff and test report to the developer for a decision.
Revision required
The restart check failed. Return the reproduction with the patch and rerun the checks after the fix.
Back to implementationHow I compare coding models
I compare coding models on representative tasks, serving performance, and GPU memory use. Every model runs the same task set on the same hardware under the same request load, and I measure single-request latency separately from performance under concurrent use.
- Task success
- Whether the patch satisfies the task, builds, and passes the relevant tests, plus a diff review for changes outside the requested scope.
- Latency
- Time to first token and time to complete the response, as a median and a tail percentile so slow requests under load stay visible.
- Throughput
- Total output tokens per second across requests at the same concurrency.
- GPU memory
- Peak VRAM use on the same hardware, with the same context length and request workload.
- Stability
- Failed and timed-out requests over repeated runs, counted against task success instead of dropped from the results.