[{"data":1,"prerenderedAt":494},["ShallowReactive",2],{"insight-evaluating-engineering-depth":3,"insights-cards":384},{"id":4,"title":5,"body":6,"category":365,"date":366,"description":367,"extension":368,"meta":369,"navigation":370,"path":371,"readingTime":372,"relatedArticles":373,"relatedAssessment":377,"relatedServices":378,"seo":381,"stem":382,"__hash__":383},"insights\u002Finsights\u002Fevaluating-engineering-depth.md","Ask About the Outage: How to Vet a Software Vendor's Real Engineering Ability",{"type":7,"value":8,"toc":347},"minimark",[9,13,18,21,24,27,30,34,37,42,49,55,61,65,71,77,83,87,93,97,197,200,204,207,247,250,254,257,296,300,303,317,320,324,327,330,333,337,340,344],[10,11,12],"p",{},"The material a software vendor shows you before a contract cannot tell you whether they are good, because a weak firm can produce the same material as a strong one. Two things do separate them: how they talk about what went wrong, and whether they can hand over evidence that was created during real work instead of for the sales process. This article is a practical procedure for collecting both.",[14,15,17],"h2",{"id":16},"why-the-standard-evidence-proves-so-little","Why the standard evidence proves so little",[10,19,20],{},"Open ten agency websites and you will see the same parts: success stories with a percentage in bold, headshots, a wall of technology logos and a diagram of a process in four or five stages.",[10,22,23],{},"None of this is dishonest. It just cannot be tested. A success story is written by the party it flatters, about a client who is often unnamed. A list of technologies says the team has used a tool, not that they have kept it running through a bad night. A process diagram describes an intention.",[10,25,26],{},"Be especially careful with headline metrics. \"Cut processing costs by 70 percent\" means nothing without the starting point, the period measured, what else changed at the same time and a client you can call. A number that cannot be checked is not evidence of results. At most it tells you the vendor is comfortable publishing numbers that cannot be checked.",[10,28,29],{},"Engineering depth shows up somewhere else. Anyone can tell a convincing story about a project that went well. Only someone who was in the room can describe, in detail, the night the database ran out of connections.",[14,31,33],{"id":32},"the-interview-three-areas-seven-questions","The interview: three areas, seven questions",[10,35,36],{},"Run this conversation with the people who would do the work, not only with the account lead. If you have an engineer on staff, bring them. You are not grading for a correct answer. You are listening for detail: names of components, times, trade-offs, the thing they changed afterward.",[38,39,41],"h3",{"id":40},"area-one-do-they-operate-software-or-only-deliver-it","Area one: do they operate software, or only deliver it?",[10,43,44,48],{},[45,46,47],"strong",{},"\"Describe your most recent production incident.\""," You want an event, not a policy. How was it detected, and how long after it started? Who got the alert? What was the immediate fix and what was the real cause? What is different now because of it? A team that runs systems has this story ready and tells it without notes. Without that experience, the reply you get is their incident management process, recited in the abstract.",[10,50,51,54],{},[45,52,53],{},"\"How does your on-call work, and who is on it this week?\""," A firm that builds and hands over has no rotation, and that is a legitimate business model if it is what you want. A firm that runs what it builds can describe the schedule, the escalation path and what counts as a page, with no preparation at all.",[10,56,57,60],{},[45,58,59],{},"\"Walk me through how a change gets from a developer's laptop to production.\""," Listen for automated tests, review by a second person, a staging environment, a way to roll back, and how often they release. If releasing is a tense, scheduled event that happens monthly, expect slow fixes. If it is boring and happens several times a week, that is a strong sign.",[38,62,64],{"id":63},"area-two-how-is-their-judgment","Area two: how is their judgment?",[10,66,67,70],{},[45,68,69],{},"\"Which of your projects failed, or came close, and what do you do differently because of it?\""," Every team that has shipped real work has one. The good answer names a decision they made, explains why it looked reasonable, says what it cost and describes what they do differently now. The worrying answers put all the blame on the client or a third party, or describe trouble that was apparently nobody's doing. If they say they have never had a bad project, they are either new or not being candid.",[10,72,73,76],{},[45,74,75],{},"\"What happens when a requirement turns out to be wrong halfway through?\""," This always happens, so the question is really about their contract model and their honesty. A workable answer: we flag it as soon as we see it, we estimate the change, and you choose. An answer to distrust: our discovery is thorough, so that does not come up. That claim will be tested within the first two months.",[10,78,79,82],{},[45,80,81],{},"\"What would you decline to build?\""," A firm with no answer will build anything a client asks for, including the feature that should never exist or the architecture that will not survive its second year. Which thing they refuse matters less than the fact that they have a line.",[38,84,86],{"id":85},"area-three-who-are-you-really-hiring","Area three: who are you really hiring?",[10,88,89,92],{},[45,90,91],{},"\"Name the engineers you would put on our project. Which other clients share their week?\""," The senior people in the pitch being replaced by juniors after signing is the oldest complaint in services. Ask for names, seniority, the share of each person's week you will get, and what happens if one of them leaves. Then write it into the agreement.",[38,94,96],{"id":95},"how-answers-tend-to-sound","How answers tend to sound",[98,99,100,116],"table",{},[101,102,103],"thead",{},[104,105,106,110,113],"tr",{},[107,108,109],"th",{},"Question",[107,111,112],{},"An answer with substance",[107,114,115],{},"An answer that should worry you",[117,118,119,131,142,153,164,175,186],"tbody",{},[104,120,121,125,128],{},[122,123,124],"td",{},"Last incident",[122,126,127],{},"Specific timeline, specific cause, a concrete change afterward",[122,129,130],{},"A description of the incident process",[104,132,133,136,139],{},[122,134,135],{},"On-call",[122,137,138],{},"Names, schedule, escalation, what triggers a page",[122,140,141],{},"\"Our support team handles that\"",[104,143,144,147,150],{},[122,145,146],{},"Deployment",[122,148,149],{},"Tests, review, staging, rollback, frequent releases",[122,151,152],{},"\"We deploy at the end of each sprint after QA sign-off\", with no mention of rollback",[104,154,155,158,161],{},[122,156,157],{},"Failed project",[122,159,160],{},"Their own decision, its cost, what changed",[122,162,163],{},"The client kept changing their mind",[104,165,166,169,172],{},[122,167,168],{},"Wrong requirement",[122,170,171],{},"Raise, price, let the client decide",[122,173,174],{},"It does not happen to us",[104,176,177,180,183],{},[122,178,179],{},"Refusals",[122,181,182],{},"A concrete example and the reasoning",[122,184,185],{},"\"We are flexible\"",[104,187,188,191,194],{},[122,189,190],{},"Staffing",[122,192,193],{},"Names and percentages, in writing",[122,195,196],{},"Roles only, \"a senior team\"",[10,198,199],{},"Notice also how the vendor reacts to being asked. Good teams tend to enjoy these questions, because the questions let them show what sets them apart. A vendor who keeps steering back to the case study page is giving you an answer too.",[14,201,203],{"id":202},"evidence-that-is-hard-to-produce-after-the-fact","Evidence that is hard to produce after the fact",[10,205,206],{},"A portfolio is selected and polished. The following items are much harder to stage, and I have ordered them from the cheapest for you to obtain to the most expensive. Work down the list and drop vendors as you go, so the costly steps are spent only on finalists.",[208,209,210,217,223,229,235,241],"ol",{},[211,212,213,216],"li",{},[45,214,215],{},"Their answer on production data access."," Ask which people at the firm would be able to open your live data, what the procedure is for giving and withdrawing that permission, and whether a log records every use of it. If every engineer holds permanent access, then no real control exists, regardless of what their website says about security. Good answers involve named individuals, time-limited grants and an audit trail.",[211,218,219,222],{},[45,220,221],{},"A description of their release pipeline."," This is the deployment question again, in writing or as a screen share. Seeing the real pipeline run takes 15 minutes and is very hard to fake.",[211,224,225,228],{},[45,226,227],{},"A postmortem they wrote."," Ask for one with client details removed. You learn whether failures are analyzed for causes or just survived. Look for a timeline, contributing factors and follow-up actions with owners, and the absence of blame aimed at individuals.",[211,230,231,234],{},[45,232,233],{},"A sample of real code, sanitized."," Have a technical person read it. Are there tests? Do comments explain why something is done, instead of repeating what the code says? Can a stranger follow the structure? You are judging habits, not style.",[211,236,237,240],{},[45,238,239],{},"A reference from a client who left."," Current clients are happy by selection. A former client tells you how the firm behaved at the end: whether they handed over documentation and credentials cleanly, and whether the system could be maintained by someone else. Most of the risk in these relationships sits at the end.",[211,242,243,246],{},[45,244,245],{},"A small paid engagement."," More on this below.",[10,248,249],{},"A single item from this list tells you more than a stack of case studies, because each was produced for a real purpose before you asked.",[14,251,253],{"id":252},"deal-breakers-when-to-walk-away","Deal-breakers: when to walk away",[10,255,256],{},"A few signals are sufficient by themselves, with no need to weigh them against anything else.",[258,259,260,266,272,278,284,290],"ul",{},[211,261,262,265],{},[45,263,264],{},"A firm quote issued before anyone has looked inside your systems."," Either the number contains a large cushion or the scope will be reopened later. Both outcomes are paid by you.",[211,267,268,271],{},[45,269,270],{},"Roles instead of people."," If nobody will name the engineers, you are buying hours from a pool. That can be acceptable, but only if it is what you were told you were buying.",[211,273,274,277],{},[45,275,276],{},"No view on your architecture."," After a serious conversation about your systems, a competent engineer has opinions and concerns. Neutral agreement with everything you say means your problem has not yet received any real thought from them.",[211,279,280,283],{},[45,281,282],{},"\"We can handle compliance.\""," For regulated data, such as health information under HIPAA, the useful answer is concrete: we have done this under signed agreements, here is how we control access, here is what we log. Vague confidence is not experience. (General information, not legal advice.)",[211,285,286,289],{},[45,287,288],{},"Reluctance to put assumptions in writing."," Written assumptions are how a scope survives its first surprise. A vendor who avoids them wants room to maneuver later.",[211,291,292,295],{},[45,293,294],{},"Pressure to skip steps."," A discount that expires on Friday is a sales tactic and has nothing to do with engineering.",[14,297,299],{"id":298},"a-simple-scorecard","A simple scorecard",[10,301,302],{},"If you are comparing several firms, keep it mechanical so charisma does not decide. Score each area from 0 to 2: 0 for no answer or a red flag, 1 for a plausible general answer, 2 for specific, checkable detail.",[258,304,305,308,311,314],{},[211,306,307],{},"Operations: incident, on-call, deployment",[211,309,310],{},"Judgment: failed project, wrong requirement, refusals",[211,312,313],{},"Staffing: named people and allocation",[211,315,316],{},"Artifacts: data access, postmortem, code, departed reference",[10,318,319],{},"Any deal-breaker from the previous section removes the vendor regardless of the total. Among the rest, low scores in operations matter most if the vendor will also run the system.",[14,321,323],{"id":322},"the-best-test-is-a-small-real-project","The best test is a small real project",[10,325,326],{},"Interviews and artifacts reduce risk, but nothing replaces watching a team do paid work on your actual problem. Before committing to the full build, commission something small with a real deliverable. A discovery phase of about two weeks is the natural form.",[10,328,329],{},"You then judge the output on its merits. Did they understand how your systems really work? Did they find something your own team did not know? Did they name risks, including ones that are inconvenient for their own proposal? Is the plan one you would follow, and could another firm follow it? A disappointing result at this stage is cheap knowledge, bought with a small fraction of the budget.",[10,331,332],{},"Two conditions make this test honest. The deliverable must belong to you, and it must be usable without the vendor who wrote it.",[14,334,336],{"id":335},"keep-geography-as-a-separate-question","Keep geography as a separate question",[10,338,339],{},"Buyers often mix \"can this team build it\" with \"where is this team located\". These are different questions. Strong and weak teams exist in every country. Location affects overlap hours, coordination cost and, for regulated data, who is allowed to touch what. Evaluate depth first, then evaluate the delivery model.",[14,341,343],{"id":342},"which-of-these-questions-would-you-most-like-answered","Which of these questions would you most like answered?",[10,345,346],{},"Put all seven to Trivo. We will answer each one, name the engineers who would be on your project and tell you directly if the work is not a good fit for how we operate.",{"title":348,"searchDepth":349,"depth":349,"links":350},"",2,[351,352,359,360,361,362,363,364],{"id":16,"depth":349,"text":17},{"id":32,"depth":349,"text":33,"children":353},[354,356,357,358],{"id":40,"depth":355,"text":41},3,{"id":63,"depth":355,"text":64},{"id":85,"depth":355,"text":86},{"id":95,"depth":355,"text":96},{"id":202,"depth":349,"text":203},{"id":252,"depth":349,"text":253},{"id":298,"depth":349,"text":299},{"id":322,"depth":349,"text":323},{"id":335,"depth":349,"text":336},{"id":342,"depth":349,"text":343},"Strategy","2026-10-12","Portfolios and logos look the same from every vendor. The interview questions, artifacts and deal-breakers that show who has really run software in production.","md",{},true,"\u002Finsights\u002Fevaluating-engineering-depth",10,[374,375,376],"two-week-discovery-deliverables","custom-software-cost-framework","nearshore-offshore-onshore-cost","build-or-buy",[379,380],"custom-software","modernization",{"title":5,"description":367},"insights\u002Fevaluating-engineering-depth","xk_j4UUrP06EucT-XmD_fvXCBkgmtVOS_jYuxW1GlBw",[385,391,398,403,408,413,421,426,431,436,441,448,450,456,462,468,474,479,484,489],{"path":386,"title":387,"description":388,"category":365,"readingTime":389,"relatedServices":390,"date":366},"\u002Finsights\u002Fbuild-vs-buy","The License Fee Is the Small Number: A Working Guide to Build or Buy","Buying software is usually right, until the product sits on top of how you make money. How to sort capabilities, price the hidden costs and decide.",9,[379,380],{"path":392,"title":393,"description":394,"category":395,"readingTime":389,"relatedServices":396,"date":366},"\u002Finsights\u002Fcloudflare-observability","The Cloudflare Signals You Need Wired Up Before Something Breaks","Cloudflare records plenty by default. Whether a person hears about a problem in time depends on choices made in a quiet week, not mid-outage.","Cloudflare",[397,380],"cloudflare",{"path":399,"title":400,"description":401,"category":395,"readingTime":389,"relatedServices":402,"date":366},"\u002Finsights\u002Fcloudflare-performance-tuning","After the Static Assets: Where Cloudflare Performance Is Actually Won","Default caching handles images, CSS and JavaScript. The larger gains sit in HTML, API responses, cache keys and origin trips that nobody tuned.",[397,379],{"path":404,"title":405,"description":406,"category":395,"readingTime":389,"relatedServices":407,"date":366},"\u002Finsights\u002Fcloudflare-waf-and-ddos","Managed Rules Know the Internet. Custom Rules Have to Know Your Application.","Cloudflare's managed WAF and DDoS rules stop generic attacks. Logins, APIs, bots and false positives still need rules written from your own traffic.",[397,379],{"path":409,"title":410,"description":411,"category":395,"readingTime":389,"relatedServices":412,"date":366},"\u002Finsights\u002Fcloudflare-zero-trust-and-integrity","Who Got In Versus What Is Running: Two Integrity Problems One Control Cannot Solve","Zero Trust verifies who reaches a system. Client-side script monitoring verifies what runs in the browser. They fail differently and need separate owners.",[397,380],{"path":414,"title":415,"description":416,"category":417,"readingTime":418,"relatedServices":419,"date":366},"\u002Finsights\u002Fcrm-adoption-and-training","Your CRM Passed Every Test and Nobody Uses It: A Diagnostic for Stalled Adoption","Low CRM adoption is rarely a training gap. How to tell whether the cause is screen friction, missing payback or management habits, and what to fix first.","CRM",8,[420],"crm",{"path":422,"title":423,"description":424,"category":417,"readingTime":418,"relatedServices":425,"date":366},"\u002Finsights\u002Fcrm-implementation-timeline","The Honest Calendar of a CRM Implementation: Five Phases and Where the Weeks Go","Provisioning a CRM takes a day. Making it the place your team sells from takes months. A phase-by-phase calendar, with the parts quotes tend to leave out.",[420,379],{"path":427,"title":428,"description":429,"category":417,"readingTime":389,"relatedServices":430,"date":366},"\u002Finsights\u002Fcrm-migration-checklist","CRM Migration Checklist: The Records Arrive, the Selling Process Gets Left Behind","A CRM migration is a process migration with a data load attached. What a plain export loses, how to choose a cutover, and a checklist for each stage.",[420,380],{"path":432,"title":433,"description":434,"category":417,"readingTime":418,"relatedServices":435,"date":366},"\u002Finsights\u002Fcustom-crm-vs-off-the-shelf","Should You Build Your Own CRM? Run the Build-or-Buy Test on Your Sales Process","Most of a CRM is a commodity you should buy. The question is whether the way you sell fits the platform's model, and what it costs when it does not.",[420,379],{"path":437,"title":438,"description":439,"category":365,"readingTime":372,"relatedServices":440,"date":366},"\u002Finsights\u002Fcustom-software-cost-framework","Estimate Before the Estimate: Sizing a Custom Software Budget on Your Own","You can't know the price before a quote, but you can know what drives it. Six cost drivers, a self-check worksheet and how to read a proposal.",[379],{"path":442,"title":443,"description":444,"category":445,"readingTime":389,"relatedServices":446,"date":366},"\u002Finsights\u002Fetl-vs-elt","Transform before or after loading? A practical ETL versus ELT decision for mid-sized teams","ETL or ELT is a question of where transformation runs and who pays for it. A practical way for a mid-sized team to choose, with the costs most plans miss.","Data",[379,447],"ai-automation",{"path":371,"title":5,"description":367,"category":365,"readingTime":372,"relatedServices":449,"date":366},[379,380],{"path":451,"title":452,"description":453,"category":454,"readingTime":372,"relatedServices":455,"date":366},"\u002Finsights\u002Fhipaa-business-associate-agreement","A BAA Is a Contract, Not a Badge: What Your Software Vendor Is Really Signing Up For","What a HIPAA Business Associate Agreement commits a software vendor to, what it leaves unproven, and what to ask and build before PHI reaches the system.","Compliance",[379,447],{"path":457,"title":458,"description":459,"category":454,"readingTime":460,"relatedServices":461,"date":366},"\u002Finsights\u002Fhipaa-technical-safeguards-checklist","Reading 45 CFR 164.312 Like a Design Document: HIPAA Technical Safeguards for Engineers","The five HIPAA technical safeguard standards mapped to concrete engineering decisions: identity, logging, integrity, authentication and encryption.",11,[379,380,397],{"path":463,"title":464,"description":465,"category":466,"readingTime":389,"relatedServices":467,"date":366},"\u002Finsights\u002Flegacy-erp-anticorruption-layer","The ERP stays as it is: building on a decades-old system through an anti-corruption layer","How to build new software on an old ERP without modifying it, using an anti-corruption layer: what goes inside, how to handle writes, when to skip it.","Modernization",[380,379],{"path":469,"title":470,"description":471,"category":472,"readingTime":372,"relatedServices":473,"date":366},"\u002Finsights\u002Fllm-in-production-confidence-thresholds","An LLM that proposes and a system that decides: thresholds, review and audit for production AI","Running an LLM in production without giving it autonomy: measured confidence, thresholds set by business cost, human review that works, an audit trail.","AI",[447,379],{"path":475,"title":476,"description":477,"category":365,"readingTime":389,"relatedServices":478,"date":366},"\u002Finsights\u002Fnearshore-offshore-onshore-cost","Cheap Hours, Expensive Projects: Pricing US, Nearshore and Offshore Teams Honestly","The hourly rate tells you little about what a project will cost. How overlap, rework, turnover and compliance change the math for each delivery model.",[379],{"path":480,"title":481,"description":482,"category":472,"readingTime":389,"relatedServices":483,"date":366},"\u002Finsights\u002Frpa-vs-api-integration","Bots on the screen or calls to the API: where RPA earns its place and where it becomes debt","RPA and API integration solve the same problem at different layers. How to choose, what screen automation really costs, and how to run it safely if you must.",[447,379],{"path":485,"title":486,"description":487,"category":466,"readingTime":389,"relatedServices":488,"date":366},"\u002Finsights\u002Fstrangler-fig-modernization","Replace the legacy system in slices: a working guide to the strangler fig pattern","How to retire a legacy system piece by piece while the business keeps running: the facade, the first slice, parallel runs, and when the pattern fails.",[380,379],{"path":490,"title":491,"description":492,"category":365,"readingTime":372,"relatedServices":493,"date":366},"\u002Finsights\u002Ftwo-week-discovery-deliverables","Paying for a Plan, Not a Pitch: What Two Weeks of Discovery Owes You","A paid discovery should leave you with documents another team could build from. The five deliverables, the work behind them and the limits to expect.",[379,380],1792618973202]