
Choosing a custom software development company is not primarily a question of who can build screens fastest. The real decision is whether the partner can turn an operating problem into a production system that your organization can own, change, secure, observe, and run after launch.
A polished portfolio can show design quality. It does not, by itself, tell you how the team handles architecture trade-offs, changing requirements, security requirements, deployment failures, technical debt, handover, or production incidents.
This guide is a practical framework for evaluating a software partner before you commit. It focuses on evidence and operating discipline rather than agency vocabulary.
For OSYSTIC's broader engineering scope, see Product Engineering & Custom Software, our delivery model, and published case studies.
1. Start with the business workflow, not a feature list
A strong partner should be able to explain the operating workflow before discussing frameworks.
Ask them to identify:
- who uses the system and in which roles;
- which actions each role can perform;
- which systems provide authoritative data;
- which workflows are synchronous versus asynchronous;
- which actions require approval;
- which rules must be deterministic;
- what happens when an external integration fails;
- what success and failure look like in production.
If the conversation jumps immediately to React, Node.js, microservices, Kubernetes, or a preferred cloud provider, the technology may be driving the design before the problem is understood.
2. Ask for architecture boundaries, not buzzwords
Architecture should make responsibilities explicit. A useful discussion should cover the application boundary, data ownership, identity and authorization, APIs, background jobs, external integrations, file/object storage, notifications, observability, deployment, and recovery.
A mature team should also explain why it is not choosing certain patterns. Microservices are not automatically better than a modular monolith. Event-driven architecture is not automatically better than a direct transaction. Managed cloud services are not automatically better when data residency or cost constraints point elsewhere.
SaaS deep dive: If the product is multi-tenant, see SaaS Architecture Decisions: Tenancy, Isolation, Data, Onboarding & Scale.
AWS's Well-Architected Framework evaluates architecture across operational excellence, security, reliability, performance efficiency, cost optimization, and sustainability. Even if your project is not on AWS, those are useful categories for testing whether a proposed architecture has considered production qualities beyond functional requirements.
3. Require security requirements in the scope, not at the end
Security should appear in acceptance criteria, architecture, testing, and deployment—not in a final penetration-test line item after the system is already built.
NIST's Secure Software Development Framework provides a common language for secure software development and explicitly notes that software producers and acquirers can use it in procurement and management discussions. OWASP ASVS 5.0 similarly provides security verification requirements that can be referenced in contracts and procurement.
Before hiring a vendor, ask how they handle:
- authentication and session security;
- role and permission enforcement;
- secret management;
- input validation and output encoding;
- file upload handling;
- API authorization;
- audit logs;
- dependency and vulnerability management;
- security testing in CI/CD;
- incident response and credential rotation.
For OSYSTIC's public control posture, see Trust & Controls.
4. Inspect the delivery system
Ask how a requirement becomes production code.
A credible delivery process should make it possible to trace:
- requirement or user story;
- acceptance criteria;
- design/architecture decision;
- implementation branch or change;
- review;
- automated tests;
- deployment;
- verification;
- rollback or fix-forward path.
DORA's current software-delivery model uses throughput and stability measures such as change lead time, deployment frequency, failed-deployment recovery time, change fail rate, and deployment rework rate. You do not need to demand elite DORA numbers from every project, but a team should be able to explain how it measures delivery speed without sacrificing stability.
5. Ask what is tested automatically
"We test before delivery" is too vague.
Ask which levels are automated:
- unit tests for business logic;
- integration tests for database and service boundaries;
- API contract tests;
- authorization tests;
- end-to-end tests for critical workflows;
- migration tests;
- security checks;
- production smoke tests.
The exact test mix depends on the system. What matters is that critical business invariants and high-risk workflows are not left entirely to manual clicking.
6. Make data ownership explicit
Ask who owns each important dataset and what the system of record is.
For example:
- Is the application database authoritative for customer status, or is the CRM?
- Does the application copy data from an ERP, or query it live?
- Who is allowed to modify a financial record?
- How are data migrations tested and rolled back?
- How are backups verified?
- How are deletions, retention, and exports handled?
Ambiguous data ownership creates duplicate records, reconciliation work, and fragile integrations.
7. Evaluate API and integration engineering
Most business software is not isolated. It connects to payments, identity providers, CRMs, ERPs, email, messaging, storage, analytics, devices, marketplaces, or internal services.
Ask the vendor how it handles:
- timeouts;
- rate limits;
- retries;
- idempotency;
- webhook verification;
- schema/version changes;
- partial failure;
- dead-letter or recovery workflows;
- integration observability.
An integration that works on the happy path in staging can still create serious production failures if duplicate requests, delayed webhooks, or third-party outages are not designed for.
Integration deep dive: See API Integration Failure Modes: Timeouts, Retries, Idempotency, Webhooks & Recovery.
8. Require observability before launch
Production support is much harder when logs and metrics are added only after the first incident.
A system should expose enough information to answer:
- Which request or job failed?
- Which user or tenant was affected?
- Which dependency caused the failure?
- What changed in the latest release?
- Can operators replay or safely retry the work?
- How quickly can the system be restored?
Ask what will be logged, which metrics will be monitored, how alerts work, and how sensitive information is excluded from logs.
9. Understand deployment and rollback
"We deploy to the cloud" does not describe a release process.
Ask:
- How are development, staging, and production separated?
- How are environment variables and secrets managed?
- How are database migrations applied?
- What automated gates run before production?
- Can a bad application release be rolled back?
- How are irreversible data migrations handled?
- Who owns the production cloud/project account?
See Cloud & DevOps Engineering for the infrastructure and delivery side of OSYSTIC's work.
10. Clarify code, infrastructure, and account ownership
Ownership should be written down before development begins.
Confirm who owns:
- source repositories;
- cloud accounts/projects;
- domains and DNS;
- databases and storage;
- CI/CD configuration;
- third-party service accounts;
- design files;
- documentation;
- deployment credentials;
- licenses and custom code.
A strong partner should be comfortable working toward client ownership and a controlled handover rather than creating accidental lock-in.
11. Ask how requirements and scope changes are controlled
Software projects change. The question is whether those changes are visible.
A healthy change process records:
- what changed;
- why it changed;
- impact on cost and timeline;
- impact on architecture or security;
- new acceptance criteria;
- who approved the change.
Without this discipline, "small changes" quietly accumulate into missed deadlines and disputes about what was included.
12. Evaluate maintainability and technical debt
Ask the vendor what it does to make the system understandable six months later.
Useful signals include:
- clear module boundaries;
- consistent naming and project structure;
- code review;
- automated formatting/linting;
- dependency hygiene;
- architecture decision records where needed;
- runbooks for important operations;
- documented setup and deployment;
- tests around critical logic.
Do not confuse maximum abstraction with maintainability. The goal is code that can be changed safely by another competent engineer.
13. Ask for a modernization opinion when an existing system already exists
If you already have software, a vendor should not assume it must be rebuilt.
Microsoft's modernization guidance recommends making the least amount of change needed to achieve the approved outcome. Replatforming, refactoring, rearchitecting, replacing, retaining, or selectively rebuilding can all be valid depending on the constraints.
For a structured decision process, see Modernize vs Rebuild: How to Decide What to Do with Legacy Software.
14. Commercial red flags
| Red flag | Why it matters |
|---|---|
| Fixed price before requirements are understood | The estimate is likely based on hidden assumptions. |
| Every project uses the same architecture | Tool preference may be replacing engineering judgment. |
| No acceptance criteria | Completion becomes subjective. |
| No explanation of code/cloud ownership | Handover and lock-in risk are unclear. |
| No test strategy | Quality depends heavily on manual inspection. |
| Security is "handled by the framework" | Application-level threats and authorization may be ignored. |
| No rollback or incident plan | The team is optimizing for launch, not operation. |
| Only happy-path demos | Failure handling and integrations remain unproven. |
15. A practical vendor scorecard
Score each candidate from 1 to 5 on:
- Problem framing — can they translate the business process into system boundaries?
- Architecture — can they explain trade-offs rather than just naming technologies?
- Security — are controls and verification requirements concrete?
- Delivery — is there a traceable path from requirement to deployment?
- Testing — are critical workflows covered automatically?
- Integration engineering — are failure, retry, idempotency, and versioning considered?
- Operations — are observability, rollback, backups, and incidents designed in?
- Ownership — are code, accounts, documentation, and handover explicit?
- Evidence — can they show relevant engineering work?
- Commercial clarity — are assumptions, milestones, acceptance, and change control understandable?
Use the scorecard to expose risks and questions, not to mechanically choose the highest number.
16. Final question: can they explain how the system will fail?
Ask the team to describe likely production failures: external API outage, database migration problem, duplicate webhook, expired credential, permission mistake, slow query, queue backlog, deployment regression, or partial transaction.
A partner that can reason concretely about failure, recovery, ownership, and observability is usually better prepared to build a system that survives real use.
Sources and further reading
- NIST — Secure Software Development Framework
- OWASP — Application Security Verification Standard 5.0
- AWS — Well-Architected Framework
- DORA — Software delivery performance metrics
- Microsoft — Cloud modernization guidance
If you are evaluating a software build, review OSYSTIC Product Engineering or send a project brief.