Roman BeltsovQA Lead / Head of QA

Experience

1. Personal project “I Hear You”

   Jul 2026 — present
   Android app with a local LLM on the phone: a voice diary with an AI coach
   Published on Google Play ↗
   Solo, built with AI agents: 646 commits, 30+ releases, ~23k lines of Kotlin, 207 requirements, ~1,000 autotestsEverything on the phone, no server: speech recognition and LLM (Gemma)
   Achievements
   • AI (2026)
     • LLM inference optimization: reply 24–28 s → 1.5–2.6 s, found the cause by measuring.
     • LLM evaluation (eval): LLM-as-a-Judge, a golden dataset, a blind pairwise judge on DeepEval — quality regression on every prompt change.
     • Guardrails: crisis messages, prompt leaks — enforced in code, covered by tests.
     • Prompt engineering: one-variable experiments; a short context prompt cut replies from 2,164 to 281 chars and won with the blind judge.
     • AI agents (Claude Code) and spec-driven development: every test traced to a requirement, 30+ releases on Google Play.
     • Product metrics in GA4 — no user text.
   More on my AI experience →

2. Kaspersky

   Dec 2023 — Jul 2026 · 2 yrs 8 mos
   Testing Group Manager (team of up to 15)
   Moscow, Russia
   IoT data gateway based on KasperskyOS (KISG) · ~30 developersKaspersky NGFW — next-generation firewall with deep traffic inspection · ~100 developers, 20–30 in testing
   Responsibilities
   • Building the testing process and test pyramid
   • Test process automation
   • Obtaining FSTEC certification
   • People management: retros, one-on-ones, performance reviews, interviews
   • Setting up test environments
   Achievements
   • AI (2025–2026)
     • RAG over internal documentation: local LightRAG (chunking, embedding, reranker) + MCP.
     • The LLM generates test design and API autotests grounded in the docs; an engineer reviews every result.
     • Test design: 4 h → 25 min per case.
     • 100% of the API (40 endpoints, 500+ tests) under autotests in a month instead of a year.
     • Bench usage monitoring (React + FastAPI) against “forgot to release”: +30% utilization, no new purchases.
     • Measured automation payoff: e2e caught a fraction of a percent of defects → checks moved down the pyramid.
   • NGFW (2024–2026)
     • Replaced one-person authoring with functional domains and sub-teams: leads grew to own functional blocks.
     • Introduced requirement tagging — each team's scope for a release became unambiguous.
     • Planned releases from real team capacity: measured where time went, evened out coverage across all functionality.
     • Rolled out test plans with suites of varying depth and product configurations instead of formal runs.
     • Built weekly quality reporting: coverage, untested requirements, defects by priority — the basis for release decisions.
     • Rebuilt the 2022–2025 backlog, set up a bench and automation for stability testing.
   • KISG (2023–2024)
     • Built test design and test plans from scratch: 100% traceability of 400 requirements.
     • Prepared the test evidence base for two FSTEC certifications — the IoT gateway (firewall) and KasperskyOS, two separate releases — passed on the first try and on schedule.
     • Built a test infrastructure of 13 benches with centralized management.
     • Cut the smoke cycle from 1 month to 2–5 days, bench deployment from 2 weeks to 1–3 days.
     • Simplified test cases: a Junior runs what used to require a Senior.
   • Team (both products)
     • Hired 7 engineers: screening, technical interview, offer decision.
     • Rebuilt onboarding: first independent test run in a week instead of 2 months.
     • Ran one-on-ones, performance reviews, retros.

3. VK Pay

   May 2020 — Aug 2023 · 3 yrs 4 mos
   QA Automation Lead (team of up to 5)
   Unified payment system for the #1 social network and media platform in Russia
   Responsibilities
   • Defining and executing the product test strategy and priorities
   • Full automation of testing processes (Quality Gates)
   • People management: one-on-ones, performance reviews, interviews
   Achievements
   • Automation and stability
     • Built functional / integration / load tests in CI and test environments.
     • Test and environment stability: from 10% to 97%.
     • e2e test run time — 6× faster.
   • Performance
     • Prepared acquiring for the AliExpress 11.11 sale: load testing (Locust), 6× throughput.
     • Set up performance monitoring: JTL Reporter, Grafana.

4. Luxoft

   2017 — 2020
   QA Automation and CI/CD Expert
   Embedding platform and internal communications service for Russian Post
   Responsibilities
   • Test automation, end-to-end CI/CD pipeline
   • Load testing with Yandex.Tank (Phantom, BFG in Python)
   Achievements
   • CI/CD and communications service
     • Cut the release cycle down to 30 minutes.
     • Built and launched an internal communications service for 130,000 users.

5. Oracle Russia & CIS

   2016 — 2017
   Senior QA Engineer
   Capital construction project tracking in Oracle Primavera
   Responsibilities
   • Test automation, framework for web and mobile apps
   • Mentoring a group of 3 in QA automation
   Achievements
   • Test automation
     • Rolled out an automation framework: manual testing time −30%.

6. vKalendare

   2014 — 2016
   Founder, own project
   Event listings across Russia — vkalendare.com

7. RADICO

   2013 — 2014
   QA Engineer
   Quality of radiation monitoring systems for the nuclear industry

8. Paragon Software

   2009 — 2012
   Software Quality Specialist
   Foundational testing and automation experience, QA team of 15

Outside work

• Smart home: Home Assistant + Zigbee, 60+ devices — I design and maintain it myself
• Travel and hiking: 18 countries, MSU hiking club, backpacking and road trips, forest music festivals
• Electric unicycle, snowboarding, drums