Outrank AI
A fraud label on a Shopify order isn't a verdict. It's a prompt to decide, before you ship, whether the margin on that order justifies the risk you're about to take. Treat Fraud Analysis Shopify gives you as a recommendation, calibrate it to your catalog, and you'll stop both shipped losses and needless false declines.
Table of Contents
Why Fraud Analysis Matters Before You Ship
In 2025, global e-commerce merchants lost 3.2% of their total annual revenue to payment fraud, according to the Merchant Risk Council's 2025 survey of more than 1,000 merchants across 35+ countries. That's not an abstract industry tax. It lands on individual orders, one parcel at a time.
One Shipped Order Shows the Math
Say a new customer checks out with a card that authorizes cleanly. Your team picks, packs, and ships the same day because speed is part of your promise. Weeks later, the cardholder's bank files a fraud chargeback and pulls the payment back.
You don't get the product back. You've also paid for shipping, packaging, support time, and the dispute process. The authorization looked like revenue. Fulfillment turned it into a loss you can't unwind.
That's why fraud analysis is an order-level decision, not a storewide percentage you check once a month. For eligible online card payments, Shopify sorts each order into low, medium, or high risk so someone on your team can fulfill, hold, or cancel it. The label supports merchant judgment. It doesn't replace it.
The Tension You Can't Ignore
Tighten your rules and you'll block some fraud. You'll also block travelers, gift buyers, VPN users, and first-time customers in a market you just launched. Loosen them and conversion improves until chargebacks start eating the gain.
The goal isn't the lowest possible fraud rate. It's the most profitable set of approved orders at a chargeback level you can sustain.
That balance starts at checkout, which is why fraud holds belong in the same conversation as Shopify checkout optimization. A friction decision that protects one order can cost you ten good ones if you never measure the trade-off. You need a default action for each risk level, signals your reviewers can read, and metrics that flag a policy that's become too tight.
What Each Shopify Risk Level Tells You to Do
Start with a default policy your operations team can run without debating every order. The table below is that starting point.
Risk level | What Shopify is saying | Default action | When to override |
|---|---|---|---|
Low | Few or no fraud indicators on an eligible card order | Capture payment and fulfill | Hold only if AOV is extreme for that customer or the market is brand new |
Medium | Mixed signals and an uncertain model score | Do not fulfill yet; investigate first | Auto-fulfill a known customer with matching history and a passed 3D Secure check |
High | Several strong indicators such as AVS or CVV failure, proxy, forwarder ZIP, or many cards | Do not fulfill; cancel or require verification | Only with a verified returning customer and a clean history on the same device and address |
What Sits Behind the Label
According to Shopify's fraud analysis documentation, the analysis combines Address Verification System (AVS) results, Card Verification Value (CVV) validation, payment-method and customer-location consistency, device and network anomalies, and repeated attempts with multiple cards. For eligible online credit-card orders, machine-learning models trained on historical transactions across Shopify stores classify chargeback risk as low, medium, or high, and those models keep improving over time.
Read the indicators, not just the color. An AVS mismatch on a gift order from a three-year customer isn't the same event as an AVS mismatch plus a proxy, a freight-forwarder ZIP, and four cards tried in ten minutes. The first deserves a quick confirmation. The second deserves a hard stop.
Why Fulfillment Is the Point of No Return
Fulfillment converts a suspicious authorization into a shipped loss exposure. Before you ship, a cancellation costs you a sale you may not have really had. After you ship, a fraud chargeback can reverse the payment while you're still responsible for the goods, the shipping, and the dispute costs.
That's why medium risk is the only tier that should routinely enter a human queue. Low-risk orders should move. Sending them to manual review burns labor and delays delivery for no gain. High-risk orders shouldn't move at all unless a reviewer finds strong counter-evidence, because the model is already telling you several signals stacked up at once.
Medium is different. Customer history, delivery risk, and an authentication result can flip the outcome there. A returning subscriber shipping to the same address on the same device is a fulfill, even with one soft mismatch. A first-time buyer with a mismatched country and an order value far above your norm is a hold until you get proof.
Don't let Shopify Protect change that logic. Coverage on eligible orders is valuable, but it's not a reason to ship every high-risk cart and hope protection applies. Preserve the fraud metadata in your review workflow, expose the underlying indicators to the people deciding, and treat the score as a recommendation you can override with evidence, not an irreversible decision. Whether this default table is too harsh for your catalog is a measurement question, and it's answerable.
Metrics That Show Whether Your Rules Help
Treat fraud reporting as a control loop, not a badge on your checkout. These are the numbers that tell you whether a rule helped or just taxed good customers.
Metric | What it measures | Benchmark or reading | Warning pattern |
|---|---|---|---|
Acceptance rate | Share of orders not flagged high risk | Platform cites 99.7% acceptance across Shopify orders, not a store guarantee | Falls after stricter rules while fraud chargebacks barely change |
High-risk-order rate | Percent of all orders Shopify marks high risk | Trend by market and campaign, not one day | Spikes only on a promo or a country you just launched |
Fraudulent chargeback rate | Chargebacks attributed to fraud | 2024 retailer average 0.52%; Mastercard MATCH flags above 1% of monthly Mastercard sales and at least $5,000 | Climbs while card-not-present approval stays loose |
Chargeback amount | Dollars in dispute | Compare with gross margin on those orders | A few high-AOV losses outweigh many small ones |
Orders canceled for fraud | Volume refused before fulfillment | Pair with estimated revenue refused | Cancels rise but chargebacks do not fall |
Shopify Protect covered orders | Eligible orders with free chargeback protection | Track protected value, not only count | Assuming every order is covered |
How to Read the Set Together
Shopify's guidance on protecting orders frames fraud analysis as an order-level decision system, with reporting built around acceptance rate, high-risk-order rate, fraudulent-chargeback rate, chargeback amount, orders canceled because of fraud, and orders protected by Shopify Protect. The acceptance rate is the share of orders not flagged high risk. The high-risk-order rate is the percentage of all orders Shopify identifies as high risk.
No single metric tells the story. Use the false-positive test: if acceptance falls and canceled orders rise but fraudulent chargebacks barely move, your new rule is refusing real customers, not fraudsters. I would roll that rule back before I'd celebrate the lower risk number.
External yardsticks keep you honest. Cybersource data reported by Shopify puts the 2024 average retailer chargeback rate at the level shown above. On the processing side, Shopify's summary of chargeback monitoring programs notes Mastercard MATCH criteria that treat chargebacks as excessive past the thresholds in the table. Cross that line and you're no longer managing order losses. You're managing your standing with the payment networks.
Run It as a Before and After Test
Record your acceptance rate and fraudulent-chargeback rate for two to four weeks. Change one rule or one Flow path. Compare over the next defined period, segmented by market, payment method, device, product, and campaign. Segmentation matters because a launch in a new country will look like an attack if you only read the blended rate.
Add review rate and recovery to the dashboard and read them over rolling periods, not a single day's score. Pair this with your broader Shopify conversion rate optimization reporting so a fraud win that suppresses conversion shows up in the same meeting. Loose approval threatens processing standing. Tight rejection suppresses repeat purchase. The metrics exist to keep you between those two failure modes.
Rules Versus the Machine Learning Score
Rules and Shopify's machine-learning risk score serve different purposes. Rules make specific signals actionable; the score groups patterns into a risk label. Treat neither as a final verdict. A false decline costs the sale and may also lose a future customer, so success means managing fraud exposure without rejecting legitimate orders unnecessarily.
Layer | What it catches | Explainable | Speed | Main failure mode |
|---|---|---|---|---|
Rules | Known patterns such as AVS or CVV mismatch, proxy or VPN, many cards from one IP, freight-forwarder ZIP | Yes, operations can explain each trigger | Immediate at checkout or in Flow | Stale logic in fragile scripts that nobody reviews |
Machine learning score | Combinations nobody wrote down | Only a low, medium, or high label | Per order | Drift as fraud patterns change |
What Rules Do Well
Rules act on known signals immediately. An AVS or CVV mismatch, proxy or VPN, repeated card attempts from one IP, or a freight-forwarder ZIP can prompt manual payment capture, a 3D Secure challenge, or a Shopify Flow path that holds or cancels an order. Card-testing protection and proxy detection also fit this layer. Each trigger has a reason operations can explain.
Rules can become stale. A blocklist nobody reviews, or an old checkout script mixing fraud checks with promotional logic, can send legitimate orders into review after a discount change. Geographic logic needs the same care: decide which location signals should trigger review rather than outright refusal, and test those signals by market before relying on them. The guide to geo filters for brand-safe reach is a marketing resource, not a guide to Shopify order risk, so it does not support this checkout decision.
What the Model Adds
Shopify's score can surface combinations that no one has encoded as a rule, then label an order low, medium, or high risk. Shopify describes its machine-learning approach on Shopify's fraud solutions page. The label is a recommendation, not the decision: what to do with a medium-risk order depends on the order's margin and the cost of a false decline, not on whether a lower fraud rate looks better.
Operating goal: approve ordinary orders quickly, authenticate uncertain ones, and stop patterns that look like abuse.
Use both layers with clear responsibilities. Rules catch known signals; the model can flag less obvious combinations but does not fully explain them, and its patterns can shift as fraud changes. A medium score therefore calls for review against the order details and your business's tolerance for loss.
Set up the Shopify Flow path so the response follows the label without treating it as proof:
Trigger on order creation so each eligible order enters the path.
Branch on the risk label rather than on one indicator alone.
Route by tier by fulfilling low-risk orders, holding medium-risk orders for review, and verifying or canceling high-risk orders before capture.
Tag each held order with a reason code that makes the review decision traceable later.
Fraud Patterns That Look Like Real Orders
Most damaging orders don't look like attacks. They look like Tuesday. Here's the review queue version of the five patterns you'll meet most.
Pattern | Typical signals | Loss if mishandled | Fitting response |
|---|---|---|---|
Card testing | Many small authorizations, several cards, one IP or device | Processor scrutiny and auth fees more than a large shipment | Velocity block; do not fulfill |
Account takeover | Stored card, new device, ship-to just changed | The order plus harm to a loyal customer | Step-up authentication, hold, contact the email already on the account |
Friendly fraud | Clean AVS and CVV, later claim of no authorization or non-receipt | A chargeback that still counts if you win | Ship with tracking and delivery proof; keep service records |
Freight-forwarder reship | Forwarder ZIP, billing country differs from ship-to | Goods leave your control on a resale | Review high-AOV international; cancel if there is no history |
Promo or bot abuse | New accounts, address variants, stacked discounts | Margin, not always a chargeback | Flow rules on discount and address clusters |
Two Patterns That Fool the Score
Card testing shows up as a burst of small authorizations across several cards from one network. Nobody's trying to steal a low-value item. They're checking which stolen cards still work, and your store pays in authorization fees and processor scrutiny. Block the velocity. Don't fulfill any of it.
Account takeover is quieter. A real customer's stored card, a new device, and a ship-to address changed minutes before checkout. Canceling the order isn't enough. You need to kill the session, force step-up authentication, and contact the email already on the account, because the person harmed here is your loyal customer.
When the Dispute Isn't Fraud at All
Friendly fraud passes AVS and CVV because the cardholder placed the order. It surfaces weeks later as a claim of no authorization or non-receipt. This is the first-party misuse end of the spectrum, and a clear explainer on how first-party fraud affects lenders helps teams see why the same behavior pattern strains merchants too.
Scale makes this worth taking seriously. Sift's dispute index report projects global chargeback volume to reach 324 million transactions by 2028, 24% above 2025, and notes merchants identify only 45% of their chargebacks as fraudulent. The rest involve confusion, delivery problems, recurring billing, or product dissatisfaction. Canceling or voiding an order doesn't prevent the cardholder's bank from filing a chargeback, an authorization hold on a legitimate statement can trigger a dispute, and every chargeback counts toward your Shopify chargeback rate whether you win or lose it.
The walkthrough below shows how flagged orders present in practice, which helps reviewers connect these patterns to real order screens.
Freight-forwarder reshipping and promo abuse round out the list. They hurt margin even when they never become fraud disputes. Your defense is evidence collected before you need it: tracking, delivery confirmation, a billing descriptor customers recognize, and clean subscription cancellation records.
How to Investigate a Medium-Risk Order
Use one queue procedure for every medium-risk order and for any high-risk order someone wants to override. Consistency is what makes the outcomes comparable later.
The Eight-Step Queue
Pause fulfillment and leave the authorization uncaptured if your payment settings allow it. An uncaptured authorization is the cheapest hold you own.
List every indicator, not just the color. Write down the AVS result, CVV result, IP versus billing location, proxy status, forwarder ZIP, and whether multiple cards were tried.
Check the customer record for prior fulfilled orders, the same device, the same address, and any subscription tenure. History is the strongest counter-signal you have.
Read the checkout context. First order, unusual order value, mismatched country, just-launched market, or steep discount each change how much proof you should demand.
Choose the extra proof. Require 3D Secure if it wasn't already applied, or send a confirmation to the email already on the account. Never rely only on a brand-new address supplied with the suspicious order.
Pick one outcome and write it down: fulfill, delay for proof, or cancel. A decision nobody recorded can't be audited or improved.
Prepare for the dispute either way. If you cancel, a chargeback is still possible. If you ship, file tracking and delivery proof before the chargeback window opens.
Log a reason code so next month's review can show whether medium-risk overrides helped or hurt.
Where the Stakes Change the Call
Subscriptions and high-AOV drops deserve extra care in step 3. A blunt cancel on a subscriber with a year of clean renewals destroys lifetime value over one soft signal. A blunt fulfill on a first-time, high-value drop order ships your largest possible loss. Same queue, different proof threshold.
Keep the middle manual until your log shows a pattern you can safely automate. Then automate that pattern, not the whole tier.
Shopify Flow should encode only the clear lows and clear highs. The moment you let a rule release held orders while a reviewer is still checking them, the queue stops being a control and becomes a race. Assign one named owner per flagged order, keep the indicators and notes with the order record, and you've also built your dispute evidence file as a by-product. That's the practical payoff of fraud analysis Shopify provides when you run it as a process instead of glancing at a badge.
Calibrate Rules to Margin Before You Tighten Them
A threshold that makes sense for a domestic repeat buyer makes no sense for a first-time international order on a thin-margin product. Split the catalog into slices before you change anything, and let your retail pricing strategies define what each slice can afford to lose.
Set a Ceiling Per Slice
Known customers under the margin-safe ceiling for that product line auto-fulfill, even with one soft signal.
First orders in the middle band go to manual review, where proof can still change the outcome.
High-risk new cards on thin-margin products get refused, because one shipped loss wipes out the margin on many good orders.
Then test the middle band instead of clamping it permanently. For two to four weeks, send part of your medium-risk orders to manual review and part to a 3D Secure challenge. Judge both arms on acceptance, review hours, fraud chargebacks, and estimated gross margin. Not fraud rate alone.
Say it plainly to your team: a lower fraud rate that suppresses repeat purchase can be the worse business result. If the 3D Secure arm accepts more good orders at a slightly higher chargeback count but better total margin, the 3D Secure arm wins.
Assign Owners Before You Run the Test
Operations owns the review queue and its turnaround. Finance owns chargeback amount and monitoring-program exposure. Growth owns acceptance by campaign, so a launch doesn't get misread as an attack.
Keep the plumbing clean or the test won't read. If checkout logic, Flow paths, and fraud metadata are tangled together, separate them first. Presidio migrated legacy Shopify Scripts into Shopify Functions through a structured transition plan, and that kind of separation is what lets a discount change stop rewriting who gets reviewed. Businesses can start by booking a call to take control of their Shopify storefront, fraud calibration included.
If your fraud rules, Flow paths, and checkout logic have grown into one tangled system nobody wants to touch, Presidio can audit the setup, separate the fraud controls from promotional logic, and rebuild them as measurable, margin-aware workflows on Shopify or Shopify Plus. Visit Presidio to book a call and turn your risk labels into decisions your team can defend with numbers.
Jamie, Presidio’s Designer, leads the practice alongside Johnnie. With over 10 years of e-commerce experience, Jay is a Shopify expert, known for crafting innovative solutions that prevent tech debt.
Jaime
Senior Product Designer, 2020
