By Steve Merrill, Founder of WRKNG Digital | October 3, 2026
Before you hire a Shopify dev or CRO agency, ask ten questions: how they measure success, their testing process, theme or headless, page speed, structured data, AI readiness, reporting, who does the work, timeline, and who owns the output. The answers tell you more than any portfolio.
I've hired dev and CRO help for stores at real scale, and the mistakes were always the questions I didn't ask up front. Ask these before you sign anything.
1. How do you measure success?
Get a number. Conversion rate lift, revenue per session, something concrete and agreed on before work starts. If they can't define a win, you can't hold them to one.
2. What's your testing process?
CRO without a real testing method is just opinions with a redesign. Ask how they run experiments, how long they wait for significance, and how they decide what to test next.
3. Theme or headless, and why?
There's no universal right answer, but there should be a reasoned one for your store. A good agency explains the tradeoff for your size and roadmap instead of defaulting to whatever they always build.
4. How do you handle page speed?
Speed affects conversion and how well crawlers read your pages. Ask what tools they use and what targets they hit. Google's Core Web Vitals are the standard worth holding them to.
5. Do you add structured data?
Product, Offer, and FAQ schema drive both search and AI visibility. A dev agency that skips structured data is building a store that's harder for Google and AI to read. Google's Product schema docs are the baseline.
6. How do you handle AI readiness?
This is the 2026 question most agencies fumble. Ask how their work affects whether ChatGPT and Google AI can read and recommend your products. If they look blank, they're building for the old web.
7. What does reporting look like?
Ask to see a sample report and confirm you get direct access to the data, not just their summary. Transparency up front predicts transparency later.
8. Who actually does the work?
Find out by name who writes your code and runs your tests. Agencies that sell with seniors and deliver with juniors are common. Meet your actual team before signing.
9. What's the realistic timeline?
Push for honest milestones, not a best-case fantasy. A partner who tells you the hard parts up front is worth more than one who promises fast and slips quietly.
10. Who owns the code and data?
You should own the theme, the custom code, and the analytics. Get the handoff terms in writing. Work you can't maintain or take with you is a dependency, not an asset.
How We Chose These Questions
These ten come from hiring dev and CRO agencies for stores at scale and watching which unasked questions caused the most pain later. We ordered them from strategy to logistics.
FAQ
Q: What's the most important question to ask a Shopify CRO agency?
How they measure success. If they can't define a win in numbers up front, you can't hold them to anything.
Q: Should a Shopify dev agency handle structured data?
Yes. It drives SEO and AI visibility. Ignoring it makes your store harder for Google and AI to read.
Q: How do I know if an agency's work will last?
Ask who owns the code and data, and get a clean handoff in writing.
Q: Do dev and CRO agencies handle AI shopping visibility?
Few do well yet. Ask directly how their work affects whether AI can read your products.
Want to know your store's AI readiness before you brief any agency? Get a free AI commerce readiness audit at WRKNG Digital's agentic commerce page.
