Our May webinar episode of When Things Go Wrong offered an eye-opening exploration of the practice known as “lab shopping.” The discussion took us behind the scenes, revealing the complexities and implications of this issue. We were thrilled to receive over 30 questions from our engaged audience both before and during the livestream, and we have provided responses to a majority of them below.
We extend our gratitude to all our panelists, including Brian Radtke of My Virgin Mary, and consultant Nichole Barber, for their exceptional insights. Special thanks to Yasha Kahn of MCR Labs for providing additional background and addressing audience questions comprehensively.
If you were the CSO for a State Cannabis Regulatory Agency, what steps would you take to prevent and/or diminish lab shopping?
Yasha: Make cannabis testing data public, announce that mislabeled products will be recalled, and start a secret shopper program with known rules (e.g. define the acceptable variance around a label claim for cannabinoids – e.g. +/- 20% as established in ASTM D8439 Standard Specification for which is aligned with the US Pharmacopeia’s Cannabis Expert Panel (Sarma et al 2020).
As a State regulator, I am interested to learn how the industry circumvents processes and the impacts to the unknowing consumer.
Yasha: If no one is watching and there are no consequences, it’s likely that someone in a market will find a way to cheat. This is true for any market/industry, including cannabis. States that have known consequences for deceiving consumers have less data manipulation.
Can a home grower, especially one who is registered as a care grower for a med card patient get flower products tested?
Yasha: This is state-specific. In FL it’s not allowed. In ME it is allowed. You should call a local lab and ask.
Role of IT in Compliance: How critical is IT infrastructure in maintaining compliance and what are the potential pitfalls for ancillary managed service providers?
Yasha: For regulators, we’ve been collecting tons of data for decades. Policymakers need access to the data and analysis tools. For operators, there needs to be automation that both reduces opportunities for errors and work required by people.
With evidence of forward progression towards federal legalization, do you foresee a national standard for testing requirements being implemented or will this fall to each state? Will labs have to be federally qualified?
Yasha: A combination. Federal will have minimum standards or suggestions, and states will (should) work together to understand how to improve policies.
The emphasis in regulating lab shopping often puts a burden on operators. How can regulations be put in place to ensure consistency amongst labs to avoid lab shopping from even becoming “necessary”?
Yasha: Investigate and enforce as soon as there are complaints. One lab offering this service to one cultivator in a state starts this snowball.
Are there market mechanisms to enforce against lab shopping? What responsibility to regulators bear for allowing this practice to blossom?
Yasha: Regulators have the responsibility of protecting consumers. If they can’t, they need to include AGs and departments of health. Brands get damaged when their companies are fined/suspended, or sued in class action lawsuits.
Colorado regulators claim that they can’t release lab results to the public because it would subject them to criminal liability, but that also prevents the public from assessing which labs are honest and which companies are lab shopping. Would you be support a law or rule that would automatically publish lab results related to finished goods offered for sale to the public?
Yasha: Absolutely.
What are states doing to bring integrity? Are there any examples of states implementing greater monitoring and checking of data? When I worked in environmental consulting we had to submit lab QA reports with all our soil and groundwater results – the state would review those and question the consultant is something was off with the QA samples.
Yasha: ME has a person dedicated to overseeing and auditing lab data. WA made their data public and a citizen data scientist, Jim MacRae, brought attention to the problem of manipulated results, regulators closed multiple labs. NV did something similar. Some states recall contaminated products. Disincentives to dishonesty need to be a part of the decision-making process for everyone (cultivators, retailers, regulators, and labs).
Open data allows citizens to audit the market and the work of the regulators.
Are we talking about a biological product where contamination is spread out where some parts are high while some parts are low in contamination? how do you avoid this issue, how do we take samples? Is there a protocol? thank you
Yasha: There are standard practices for sampling. While the lack of this may be a problem, it’s minor compared to the decisions made by labs to manipulate results.
It seems that pesticides always get lost in these conversations. They should be at the top of every lab discussion. The pesticide levels we see in a lot of biomass is terrifying, even scarier is the fact that these compounds breakdown into things like Hydrogen Cyanide and Hydrogen Chloride when combusted. I see very wide variances in detection levels depending on the lab we use. What efforts are being made to standardize pesticide testing methodologies? How prevalent is lab shopping for pesticides, are there any stats to back it up?
Yasha: There will be a very informative discussion about this exact topic in a few weeks at the ASTM meeting. It is very important and under-discussed.
The manipulation that Yasha described is grounds from fines, penalties, even the possibility of criminal violations in sampling and testing in other industries. Is this inadequate enforcement?
David: Under-enforcement and under-resources are definitely part of the issue. It also varies greatly based on state by state. For example, does a reguator have enough evidence to go after them? Are the rules clear that there is a violation? The lack of documentation in the industry makes it difficult to prove. It’s also so prevalent in many markets that it’s a game of whackamole, and resource intensive.
Are there any trends in consumer awareness and education that are calling for quality and away from simply THC content?
Yasha: yes, but a few people trying to educate are nothing compared to the millions spent by industry to promote their 40%s.
Are there insights on how the regulators in the markets approach these labs and those who are corrupted? Let’s be real, as much as the business owner tries to be self-regulated/ethical there is no benefit in playing by the rules if its an acceptable practice within the state
Yasha: Yes, labs have been closed for manipulating results. NV and WA are best examples. Some states are currently rolling up their sleeves to do so. Complaints lead to investigations lead to consequences. This works. States that have no recalls and no fines/lab suspensions seem to have the most result manipulation.
What states is Yasha talking about? Let’s talk up the ones working to bring integrity!
Yasha: Don’t know which statement this is referring to, but the states that have fined or closed most labs are WA and NV.
Shouldn’t the lab homogenize the sample and create an average of all flower collected. Samples are purchased in packages and sold that way, no one is selling single buds, studies looking at individual packages and the average, they line up very closely, from data sub 10% relative when looking at it at property sample and homogenized flower
Yasha: Absolutely. Lab studies and big data analyses support this.
Would labs & METRC being more associated an option? Why should R&D tests not be associated with the product in METRC when compliance testing is?
Yasha: This is state and regulation-specific. Some states require more data to be shared. An argument against requiring R&D samples to be submitted is that labs may not want anyone, including regulators, to know what they are working on, and it’s not relevant to public health. Data submitted can be made public through FOIA.
Pointing to heterogeneity is a cop-out to deflect from rampant manipulation.
Yasha: Agreed. Heterogeneity is quantifiable and minor compared to the real problem.
Who is the consumer? The prescribing doc/clinic/pharmacy or is it the patient? How can a patient be informed to make a decision on what is safe or not?
Yasha: I think the customer needs to be able to trust labels in cannabis, as in any other purchasing. Consumers harmed (health of financial) by dishonest/inaccurate labeling should be reimbursed by those who made money because of the dishonesty. I know this doesn’t directly answer the question…
How much quality is sacrificed during Radsource decontamination, e.g. cannabinoids and their degradation and terpenes and the loss by treatment? if there is any decrease in cannabinoids and terpenes while reducing microbial load?
Brian: Great question. If the company that is running the Rad Source understand the proper temps to run the equipment than no degradation should happen at all. There are levels to all things, including running decontamination machines. People use RF machines all the time and don’t realize that it’s heating up the product and decarboxylating it to remove the moisture from the product.
Should THC and other cannabinoids not be given in a range instead of an average value of the sample (sampling small flowers for lower end and big nugs for the high value and state the [low-high value range] on packaging – better to have consumers know product comes in a natural range of secondary metabolites in lower to top flower?
Yasha: I think it’s a good idea. If we have ranges at 25-30% and 30-35%, there will still be cheating. If the ranges are low (Low (under 5%), Medium (5-15%), high (over 15%), Consumers can still plan their sessions and potency inflation would likely reduce significantly if not completely. Or we can add a tax per mg or % of THC.
Cannabis buds on different locations of the same plant have a large potency variation. I’d like to better understand how this factor impacts interlab differences in potency results. Thanks
Yasha: Cannabis flower is inhomogenous, but we know how inhomogenous it is. If samples are sent to two different labs, results may slightly differ if they are nugs from different plants or parts of plants. However, the law of large numbers states that the average of the results obtained from a large number of random samples converges to the true value. If both labs are honest, their averages, if testing many samples for the same cultivators, will be the same (or similar).
If averages drastically differ, this can not be due to cannabis’s heterogeneity but to a difference in lab procedure.
Do most cannabis cultivators/processes NOT qualify the laboratories they use and NOT have a quality focused person/group that reviews the results generated by the lab before accepting on face value? It just seems like an ethical/integrity issue for both the cultivator/processer and the lab who should have valid methods and/or not intentionally inflating lab values.
David: In our experience at The GMP Collective, properly qualifying the laboratories is not common. I attribute that to multiple factors. In some cases, there are a lack of options in the marketplace (e.g. 1 or 2 labs in the state). Historically, and especially for smaller operators, it is rare to have anyone with a Quality Assurance background on payroll/staff at a cannabis operation. More states are requiring GMP or quality systems which means you need to have a quality role, but this again points to a lack of education by operators.
What is your opinion of whether standardizing lab methods would work or not can anyone comment on how it worked in CA for flower?
Yasha: The pros: Standardized methods allow regulators to learn just a few methods and audit all labs. Regulators who must learn many unique and proprietary methods and their potential deviations will not be able to audit labs as well.
The cons: decrease in innovation. We can’t add new cannabinoids, or make the test faster or cheaper.
CA did this, and it seems to have worked for some time. Then labs had to make the decision of “to cheat or not to cheat” and my assumption is that many started to again. The alternative to cheating is for the decision makers to lay off their peers. But, because of the standardization, CA may now be able to audit labs better than before, which may lead to more consumer protections and honesty.
Standardization of methods without adding auditing will not change anything. A non-standardized but validated method is sufficient and accurate. Different methods, if validated, will yield similar results. The lack of standards is not the problem. The problem is dishonesty. However, standardization of methods may help regulators keep operators honest.
What are some innovative approaches to mitigating lab shopping that you’re seeing in the market? What about making the connection between the cultivator/processor and the laboratory anonymous? If one group doesn’t know who the other group is that could potentially mitigate fraud.
Yasha: If lab selection is based on results, the problem will persist.
How can we implement checks and balances that achieve a more reliable system and reduces opportunities to game the system? Implementing a standardized water activity range for instance could help with weight stabilization and therefore potency consistency. What are the MVP standards that should be put in place?
Yasha: make all cannabis testing data public. Data scientists will find the fraud and call it out. It also would allow regulators from different states to understand their state’s data better. And, perform off the shelf testing with results made public.
Part of the ethical fix is creating a culture of compliance starting at the highest level
David: Could not agree more. And that also requires trust. Not just internally (within a company), but between stakeholders as well (e.g. regulators, cultivators, laboratories).
Do labs typically act on their own to inflate THC results or do they act at the behest of the cultivator/processor? Or, both?
Yasha: Typically in a market, a single lab starts offering inaccurate results. One cultivator testing with this lab pressures all other cultivators in the market to do the same. You, a seasoned cultivator are proudly getting 26% on your flower. A cultivator down the street who’s growing for their second season is averaging 34% with an untrained staff. And your neighbor never fails for mold while stuffing the grow room with 2x as many plants as you.
You’ll tell your lab that you want higher potency numbers and fewer fails. The lab will have to decide whether to lose your business or comply.
Other than that first lab that made the decision, I don’t know how to answer your question.
Thanks again to everyone who tuned in. Catch up on all of our past webinar episodes on our YouTube channel, and be sure to register for our next webinar in June.
