AI work

LLM Inference & Evaluation

I build and maintain Biexce’s internal platform for coding models. My work covers model serving, benchmark design, and agent workflows that connect to developer tools.

Coding model evaluation

Use the same coding tasks and project context for each configuration so differences in output are easier to interpret.

Serve each model through vLLM. Measure when each response starts and ends, along with aggregate throughput and GPU memory use.

Compare the generated changes with the task requirements. Review serving metrics and failed runs alongside code quality.

Role
AI Engineer · Biexce
Scope
Internal developer platform
Stack
Docker · vLLM · Bifrost · MCP

Serving models on GPUs

Developer tools call a shared API through Bifrost. The gateway routes requests to vLLM, which serves the model in a Dockerized runtime on GPU servers. I monitor the services and troubleshoot model and infrastructure failures.

Multi-agent coding workflow

I build multi-agent workflows for planning, implementation, testing, and code review. MCP connects the workflow to Unity, Git, and internal documentation. The planner turns a task into acceptance checks, and the workflow carries the proposed diff and test report into review.

Illustrative Unity task

Wait for the last enemy before advancing the wave

Define what the fix must do

The planner reads the task and the project’s wave rules. It identifies the wave controller and match reset as the parts to inspect, then turns the bug report into acceptance checks.

Acceptance checks

  • Keep the current wave active while enemies remain.
  • Advance once after spawning finishes and the last enemy is gone.
  • Clear the pending transition when the match restarts.

Change the wave transition

The coding agent uses Unity project context and the Git working tree to locate the existing transition logic. It makes a scoped C# change and hands the diff to the test stage.

Proposed Git changes

  • Gate the transition on completed spawning and zero active enemies.
  • Guard against duplicate advance requests.
  • Cancel the pending transition in the match reset path.

Exercise the changed behavior

The test stage checks that the project compiles, then uses Unity Play Mode tests to exercise the wave lifecycle. If a check fails, the expected and observed behavior go back to implementation.

Sample Play Mode results
  • Spawning continues · wave stays activePass
  • Enemies remain · wave stays activePass
  • Final enemy removed · wave advances oncePass
  • Restart · pending transition clearsPassFail

The previous wave’s transition still runs after restart. Send that reproduction back with the failed check.

Back to implementation

Review the diff and test results

The review agent checks the patch against the original task, inspects the test report, and flags changes outside the requested scope. The developer gets the diff and the evidence behind the recommendation.

Review decision

Ready for developer review

The wave checks passed. Send the scoped diff and test report to the developer for a decision.

Revision required

The restart check failed. Return the reproduction with the patch and rerun the checks after the fix.

Back to implementation

How I compare coding models

I compare coding models on representative tasks, serving performance, and GPU memory use. Every model runs the same task set on the same hardware under the same request load, and I measure single-request latency separately from performance under concurrent use.

Task success
Whether the patch satisfies the task, builds, and passes the relevant tests, plus a diff review for changes outside the requested scope.
Latency
Time to first token and time to complete the response, as a median and a tail percentile so slow requests under load stay visible.
Throughput
Total output tokens per second across requests at the same concurrency.
GPU memory
Peak VRAM use on the same hardware, with the same context length and request workload.
Stability
Failed and timed-out requests over repeated runs, counted against task success instead of dropped from the results.