1. Coupon Revenue Could Not Tell Me Whether Product Seeding Worked

This article explores a deceptively simple question: How do you measure whether sending free products to influencers actually increased sales? The answer is harder than it looks, and rethinking the problem changed what I built.

At first, the approach seemed obvious-use AI to rank influencers by follower count and engagement, then send products to the highest scorers. But measuring success revealed a hidden problem. Coupon-tracked orders told me who received the product, not whether that product caused the sales. The same customer might have bought anyway. The coupon code might have spread to other channels. Concurrent ads or promotions could have driven the same orders.

That gap between “orders attributed to the influencer” and “sales caused by seeding” is the core problem. To bridge it, I needed a comparison-a group of similar influencers who did not receive products yet. By comparing orders from creators who got the product to orders from creators still waiting, I could estimate the true incremental impact of seeding. That is the title’s thesis: without a comparison group, no measurement is possible.

The resulting tool does not predict revenue with high precision. Instead, it enforces rigor. It blocks risky ad copy before it ships, requires company-approved sources only, prevents users from re-rolling assignments when results look unfavorable, and refuses to call a result “successful” when the sample is too small. It admits what it cannot prove.

This project used synthetic data and public information for testing, not real company records. I need to be direct about this: I did not verify any real sales increase. This article is not a proof of impact. Instead, it records building the procedure and safety gates first, so that real validation can start on a foundation that already blocks the common shortcuts and mistakes.

Designing Product Seeding with AI

The numbers inside the product are synthetic or drawn from public data. Every candidate, order, and revenue figure shown in this article or its screens is either generated for testing or comes from public sources. None of those figures represents any specific company’s actual creator list, order history, or sales performance. The product data exists to test the design.

The challenge asked me to improve product seeding with AI. My first instinct was to rank creators more accurately. As I built the product, I found that the harder questions came before and after ranking. Before sending anything, the workflow had to stop risky ad copy. Afterward, it needed a comparison that could help explain why orders changed.

That discovery changed what I was building. A creator score could still help allocate a limited test budget, but it could not stand in for evidence that seeding caused new sales.

I Started by Trying to Predict Which Creators Would Sell

My first interface looked familiar. It collected follower counts, average views, and engagement rates, then assigned each creator a score. A marketer could compare many accounts quickly and send products to the highest-ranked candidates. Even without a sales model, normalizing public metrics could reduce manual work.

The problem began when I tried to turn that score into a revenue forecast. I had no internal sales history, so I could not train or validate a model against the number of paid orders each creator had generated. High views and engagement showed attention. They did not show that the attention had become company revenue. Adding decimal places to projected return on ad spend would have made the estimate look more certain without giving it a stronger basis.

I also checked the original December 2019 file from a public cosmetics-store dataset for a behavioral baseline. The file contains 3,533,286 event rows from a public cosmetics-store dataset for December 2019. Those rows record public behavior such as views, cart actions, and purchases. They are not company data, and they cannot replace the missing internal sales history.

I stopped trying to make the forecast look more precise. I kept the creator score only as a way to decide where a limited experiment budget should go first. A separate comparison would evaluate what happened after execution. The product now treated prioritization and measured results as two different things.

Coupon Orders Did Not Tell Me Whether Seeding Worked

Dedicated coupons and tracking links still mattered. They let the team count orders associated with a creator-specific path. But coupon- and link-tracked revenue is not the same as incremental revenue attributable to the intervention.

A customer who already intended to buy might use the coupon. A code might spread through another channel. Search ads, messages to existing customers, or another campaign running at the same time could also affect orders. If I counted every tracked order as seeding impact, I would confuse the path attached to an order with the reason that order occurred.

The missing quantity was the outcome without seeding. A coupon cannot create that comparison by itself. I needed a group of similar creators who would not receive products until the measurement window closed. They were not permanently excluded. They formed a delayed-start comparison group that could begin later.

The experiment compared creators receiving products now with creators who had not yet received them over the same period. Google Ads guidance on experiments likewise describes randomized exposed and unexposed groups as the basis for measuring incremental impact rather than relying only on attributed activity.

A comparison group does not remove every source of bias. Creators may reach overlapping audiences, and concurrent campaigns can contaminate the result. Execution may also depart from the assigned plan. Even with those limits, the design records what was compared instead of treating coupon orders as a causal answer.

I Checked Ad Copy Before Scoring Creators

Some candidates should not enter the ranking at all until a person reviews the proposed copy. A missing sponsorship disclosure was one reason to stop. A disclosure paired with wording such as not an ad was another because the surrounding copy contradicted the disclosure. I applied the same pre-publication check to disease-treatment language and claims that were not in the company’s approved material. A high audience score could not offset those risks.

For economic-interest disclosure, I referred to the National Law Information Center’s recommendation and endorsement advertising review guideline . The relationship should be presented where consumers can notice and understand it, close to the recommendation, and the rest of the post should not weaken or contradict it. For copy that could make a cosmetic sound like a medicine, I used Article 13 of the Korean Cosmetics Act as the reason to screen treatment-oriented language before publication.

This was a pre-publication risk screen, not legal advice. Matching a rule or an approved phrase does not guarantee compliance. Jurisdiction, product classification, and the full context of the post still require human review. The product’s job was narrower: stop risky candidates before their reach or engagement scores moved them to the top.

Putting this check first changed the interface. Instead of opening with the highest scores, the workflow first showed which disclosure was missing and which claim fell outside the approved material. It also recorded why a candidate was paused so a marketer could revise the copy and submit it for review again.

I Used Only Data the Company Had Approved

A safety check is easy to bypass if the caller can type both the claim and its supposed source. I created a company-approved registry and required product copy to match an approved entry exactly. The same registry supplied a snapshot containing the approver, approval time, and official source. A request could not invent its own copy and approval evidence and present both as trusted input.

Creator metrics needed more than a number as well. The workflow checked a stable creator ID and the time at which the value was collected. It compared the metric name and value with the supplied source file, then calculated a SHA-256 value for that file. If even one byte changed, the calculated value changed too, giving the system a way to detect a later modification.

SHA-256 only supports that limited claim. A matching value can show that the file presented later is the same file that was recorded. It cannot show that the platform or an authorized employee issued the file. Someone could fabricate a file, calculate its hash, and pass the format check. A real deployment would also need an access-controlled source export or a signed export.

I kept this distinction visible in the result. File-content and fingerprint checks belong to the system. Source authenticity requires information about the issuing path and the issuer’s authority. The interface did not combine those two states into one approval label.

I Randomly Split Creators into Seeding and Comparison Groups

Only creators who passed the copy and source checks entered random assignment. The product split them into a treatment group receiving products now and a delayed-start comparison group. I also had to protect that assignment from repeated attempts to obtain a preferred result.

If the interface reveals the result before saving it, a user can rename a request and run it again whenever the split looks inconvenient. I reversed that order. The system had to record the assignment before revealing it.

Changes to display-only fields, such as a campaign name or goal description, returned the existing assignment. Changes to inputs that could affect the experiment, including the candidate set, budget, or measurement window, did not create a fresh split. They produced a conflict and stopped the request. That behavior reduced assignment shopping through small input changes.

Budget feasibility had to come before the assignment record as well. The system checked whether every possible treatment combination could satisfy both the minimum execution amount and the per-creator cap. If the budget could not support a valid allocation, the request stopped without saving an assignment. Once the caller corrected the conditions, the first valid request could receive the first recorded split.

For the article, I used a simple absolute rate-difference estimate:

Estimated seeding effect = treatment-group order-event rate - comparison-group order-event rate

The groups can have different sizes, so subtracting raw order counts would be misleading. The estimator compares rates instead. I did not treat this formula as the only definition of incremental measurement. Advertising measurement systems may adjust for missing outcomes and report relative differences or confidence intervals. Here, the pre-specified absolute rate difference made the decision structure of a small experiment easy to inspect.

Five Creators Were Not Enough for a Conclusion

Five synthetic candidates passed the safety and source checks in the demo. Random assignment put three in the treatment group and two in the delayed-start comparison group. Even if the difference between their order rates looked large, five creators could not support a performance claim.

The sample-size calculation used an 8% baseline order-event rate, a minimum detectable absolute difference of 4 percentage points, a significance level of 0.05, and power of 0.8. Under those assumptions, the approximate requirement was 883 creators per group. That is not a fixed threshold for every campaign. A different baseline, target difference, measurement period, or allocation would change the required sample.

I included a two-sided Fisher’s exact test because it calculates the possible tables directly when counts are small. I paired it with a Newcombe interval, which combines Wilson intervals for the two group proportions to show a range for their difference. The interface presented the test result alongside the sample plan and the interval rather than treating one p-value as the decision.

When I fed the synthetic result into the analyzer, its actual output was PILOT_ONLY_DO_NOT_CLAIM. In product terms, that means pilot only, no performance claim. The analyzer returned the same state whenever the pre-specified sample condition was not met, even if the observed difference looked favorable or a statistical test happened to look significant.

That warning did not weaken the demo. It stopped a five-creator result from being presented as a success story. The smaller the sample, the more direct the product needed to be about what the numbers could not establish.

I Asked AI How the System Could Be Gamed

I did not ask AI to decide which creator would produce the most revenue. With no validated sales history, it could still produce a confident ranking, but that ranking would not solve the evidence problem. I used AI to generate attack questions instead.

Could a caller fabricate the approved copy and the approver together? Could renaming a campaign trigger repeated random assignments? Could someone submit past dates and close the measurement immediately? These questions became product contracts. I specified which inputs to reject, when a failed request should leave no assignment record, and when the system had to reuse an existing split.

I also used a separate AI session with no implementation context. That session repeated the attacks and checked whether the behavior described in the documentation matched the behavior in code. Keeping review context separate reduced the chance that the same conversation would explain and approve its own decisions.

AI still did not provide an independent guarantee of correctness. People had to choose the approved claims, approvers, budget, outcome metric, and experiment conditions. The separate review narrowed the material a person needed to inspect. It did not prove that operational data was authentic or that the experiment design was valid for every campaign.

I Tried to Break the Design with 32 Bad Inputs

I turned the attack questions into 32 automated tests. A successful request was not enough. The tests deliberately supplied altered records, unsafe copy, infeasible budgets, and attempts to obtain a more convenient assignment.

I reran the tests from the original submitted project and got 32/32 tests passed. This is not evidence that a company campaign succeeded. It says that the implementation applied its specified stop and reuse behavior to those 32 attack cases.

The tests also made me narrow several statements in the documentation. I had first described SHA-256 as if it verified the source. The test boundary showed that it only checks whether file content changed. I also separated a server receipt time from publisher authentication. A receipt time can prevent a caller from backdating submission, but it cannot prove who created the result file.

Recording what the tests did not verify took more work than reporting that they passed. That boundary was part of the product behavior, not a footnote added after the demo.

What Validation Required from Real Campaigns

I did not execute a real company seeding campaign, and I did not have access to the company’s raw sales records. I could not claim that the intervention increased real sales. I also could not show that the same design would produce a similar result across different campaigns.

A result file can match its SHA-256 value while leaving its publisher unknown. A server receipt time can prevent backdating without authenticating the orders inside the file. The system also cannot automatically correct overlapping creator audiences or cases in which execution does not follow the assigned plan.

The next real step would be to register one campaign’s candidates, outcome metric, measurement window, and analysis method before execution. Results should come from an access-controlled source-system export or a signed export. That would connect the recorded assignment to observed outcomes while preserving information about where the result came from.

One properly run campaign would still not justify broad claims. Different campaign conditions require repeated measurement, and each result should report its uncertainty. This project built a structure for starting that validation. It did not produce verified sales impact.

Changing the Question Changed the Product

I began with a prediction question: which creator will sell the most product? If I had kept that question, the product would have centered on creator scores and projected revenue. Without internal sales data, a more polished ranking would only have made an estimate look like a confirmed fact.

The revised question changed the main workflow. First, decide whether execution is safe enough to proceed. Then compare the group that received products with the group that waited. Ranking became supporting information for choosing experiment candidates, and results without an adequate comparison or sample did not receive a performance label.

This change also altered when the product should answer. A prediction tool tries to produce one number before execution. My design records reasons to stop and the assignment criteria before execution. It calculates the group difference only after the measurement window ends.

The leaderboard is not the record I most want to see next. I want to inspect a real campaign that followed its pre-registered conditions and kept the same reporting standard when the result was disappointing. At that point, the product would no longer be a tool for guessing who might sell. It would help me review whether the campaign was safe to run and whether the recorded result supported the decision.