Aquatic toxicity part II: precision and accuracy of classifications
INTRODUCTION
California State Toxicity is a notoriously difficult waste classification to master for retailers and waste professionals.
Classifying State Toxicity correctly can serve as a major benefit to waste generators by reducing regulated waste volumes, hauling costs, and associated fees.
California’s Department of Toxic Substances Control (DTSC) has published aquatic toxicity results for several hundred consumer products. The study evaluated the products’ aquatic toxicity and assigned labels of “Pass” to products that are non-toxic to the aquatic environment and “Fail” to products that are toxic to the aquatic environment.
While the traditional aquatic toxicity test is a widely accepted testing standard, it necessitates the usage of live animal testing on fish. More and more, this practice is being viewed as unethical, costly and unnecessary by manufacturers and consumers alike.
Computational methods of toxicology testing are increasingly being considered as more effective and humane alternatives to traditional methods.
To verify this claim, Smarter Sorting applied an automated toxicity calculation to the same DTSC dataset and evaluated the efficacy of our calculations in determining California aquatic toxicity.
The results are promising:
- Precision = 100%
- Recall = 77%
- Specificity = 100%
- Balanced accuracy = 89%
Keep these numbers top of mind – we’ll be addressing them soon and explaining exactly what they mean for the aquatic toxicity determination, and more importantly, for fish.
BACKGROUND
Determining State Toxicity in California requires the evaluation of many criteria, which include carcinogenicity, toxicity, flammability, and corrosivity to name a few.
The DTSC outlines the myriad ways in which a waste qualifies as a California State Toxic on their website. One of these criteria is called Acute Aquatic Toxic, where a “waste is hazardous by aquatic toxicity if a 96-hour LC50 is less than 500 mg/L.”
The DTSC does not provide guidance on the methodological evaluation of aquatic toxicity as it pertains to consumer products. Whether computational methods are permitted or not is left unanswered. The text suggests that each and every consumer product should undergo live animal testing to evaluate the aquatic toxicity and determine if the product is hazardous in California.
Live animal testing a substance for aquatic toxicity is not a trivial process. The EPA claims that each test could cost well over $14,000. These tests take time and resources that many retail businesses simply do not have.
When faced with onerous or complicated state hazardous criteria, many retailers will skip the hazardous evaluation process all together. Instead, they opt to just consider all potentially hazardous waste as hazardous. In this manner, many retailers are over-regulating their waste streams, and overpaying for expensive waste treatment in the process.
Smarter Sorting has automated the calculation of toxicity-based state regulations in the past with great success. In our Computer Bits vs. California Fish article, we demonstrated the difficulties associated with determining aquatic toxicity classifications for consumer products, and the accuracy of estimating toxicity calculations in a sample product.
We also demonstrated how Smarter Sorting’s ability to make complex state regulatory calculations typically reduces the amount of regulated waste retailers generate.
For this article, we applied our aquatic toxicity methodology to a larger sample of 100 test products to determine the accuracy and viability of computerized testing as an alternative to live fish testing.
In the following analysis, the terms “non-toxic” and “pass” are synonymous and indicate aquatic toxicity values greater than 500 mg/L. Conversely, the terms “toxic” and “fail” indicate aquatic toxicity values of less than 500 mg/L.
DATA
Check out the original dataset used in our analysis.
The 100 sample products were randomized and satisfied the following criteria:
- Smarter Sorting has the product Safety Data Sheet (SDS) on file
- At least one of the Eurofins Calscience or Aquatic Testing labs had conducted a pass or fail study on each product
Learn more about sample products and results!
METHODOLOGY
Smarter Sorting used the Acute Toxicity Estimation (ATE) to arrive at our aquatic toxicity estimates. The chemicals and associated concentrations for each product were derived from SDSs and ingredients labels. For ingredients derived from product labels in lieu of the SDS, we deduced ranges of potential concentrations based on regulatory requirements for listing chemicals on SDSs and ingredients labels. For more information on how concentrations were estimated, refer to our previous paper on calculating aquatic toxicity.
Using the calculated aquatic ATE, we assigned products with ATE values greater than 500 mg/L a score of PASS. For products with ATE values of less than 500 mg/L we assigned a score of FAIL. This threshold is based on information listed on the DTSC website. (It should be noted that the published study labels several products as FAILS despite having LC50 values greater than 500 mg/L (e.g., sample numbers 180, 220, 296, 374, 389)).
When a product received both a PASS and FAIL from our calculation and the standard lab test, we considered the product a FAIL.
RESULTS
Smarter Sorting Aquatic ATE Test vs. Traditional Lab Testing
Data Analysis Metrics
DISCUSSION
To make sense of the data, let’s first unpack the results from Smarter Sorting’s aquatic ATE test as compared to traditional lab testing:
- In 47 cases, we said an item was non-toxic and the labs agreed
- There were zero cases where we said an item was non-toxic and the labs disagreed
- In 39 cases, we said an item was toxic and the labs agreed
- In 14 cases, we said an item was toxic and the labs disagreed
Smarter Sorting Aquatic ATE Test vs. Traditional Lab Testing
Let’s break down the data science jargon to understand how these metrics are applied to their respective probabilities:
- Precision of 100%: Given Smarter Sorting’s classification of Non-Toxic, the likelihood that the product is actually non-toxic is 100%.
- Recall of 77%: Given a laboratory classification of Non-Toxic, the likelihood of Smarter Sorting’s classification being non-toxic is 77%.
- Specificity of 100%: Given a laboratory classification of Toxic, the likelihood of Smarter Sorting’s classification being Toxic is 100%.
- Balanced Accuracy refers to the accuracy of Smarter Sorting’s Toxic versus Non-Toxic calculations normalized to account for the number in each class.
One important to note is that Smarter Sorting’s calculations appear to be more conservative (more likely to indicate that a product is Toxic) than laboratory testing.
CHANGING THE STATUS QUO
By working with several retailers, Smarter Sorting has observed that regulated state waste identification remains a persistent and costly problem. The status quo for many retailers is to “play it safe” and lump nearly all non-RCRA wastes together with regulated state wastes.
In other words, retailers are over-regulating their waste. They do not have the time nor resources to evaluate every state’s hazardous waste criteria to determine if some of their wastes are non-hazardous. This is precisely where computational methods of toxicology stand to help waste professionals and retailers the most.
THE SMARTER SORTING ADVANTAGE
Smarter Sorting’s aquatic toxicity calculations detect aquatic toxicity as defined by DTSC when sufficient ingredient information is available.
Using Smarter Sorting’s aquatic toxicity calculations, complicated and burdensome waste regulations can be calculated instantly, accurately, and without harming any fish. In the process, retailers may see a dramatic decrease of up to 47% in their state regulated waste.
Leveraging Smarter Sorting’s automated classifications for California State Waste could change the way regulated state waste is classified and ultimately handled in California. Most importantly, it could eliminate the need for animal testing in the evaluation of hazardous wastes.
That’s a huge win for retailers, waste professionals, and fish everywhere.
Related Posts
.png)
What is gravy!?! An unconventional dive into what makes a product
As you dig yourself out of that post-holiday workload this week - we invite you to take a quick pause and join us on a journey that began during Thanksgiving week. In our SmarterX internal company chat, a seemingly trivial question about gravy ignited a discussion that led us on a somewhat ridiculous but undeniably important exploration.
Who defines if gravy is a liquid or a solid? What is the basis of that definition? When you pick up a gravy product, where did it come from? And how does where it comes from impact its value or characteristics? How does all of that impact how it’s handled across the supply chain? Each tablespoon of gravy - as with any product - comes with a million questions and answers.
These questions aren't just about gravy; they're about the very essence of products and substances we encounter every day. We had some laughs along the way, but we promise - our explorations will get you thinking 🧠:
It all began with a simple question: ‘What is gravy?’
Some of us asked ChatGPT directly. If you’re wondering about it’s traditional, textbook definition, here ya go:
Gravy, traditionally understood as a sauce made from the thickened and seasoned juices of cooked meat, has a rich history in the culinary world. This definition is consistent across various sources, with the Cambridge English Dictionary describing it as a sauce made with meat juices and flour, served with meat and vegetables. The Collins English Dictionary adds that it can also be a sauce made by thickening and flavoring the juices that exude from meat during cooking. Similarly, Dictionary.com defines gravy as the fat and juices that drip from cooking meat, often thickened and seasoned, and used as a sauce for various dishes. Wikipedia expands on this by noting that it's often made from the juices of meats that run naturally during cooking and thickened with substances like cornstarch for added texture.
Does that bore you? Yeah, us too. So we went deeper.
As we dove in as a team, we quickly found that the diversity of products labeled as "gravy" in our own database at SmarterX added several extra layers of intrigue to the discussion.
Gravy is not just a simple sauce; it's a product with a rich and varied history, different types, and more. It can be found in various forms, from pet care products to warm sauces, and it even blurs the line between food and pharmaceuticals.
Team members from across the organization got creative leveraging Artificial Intelligence to shape and mold our data on gravy. It started with our SmarterX data on ~4000 products tagged as 'gravy' in our systems... and evolved from there.
.png)
.png)
The Gravy Classifier ---> Visualizing the “WHAT” of’ what is gravy?’
Within minutes, ChatGPT could determine whether a product in the database was gravy or not, based on simply the product’s name and item type. While we went down several interesting rabbit holes on gravy as a team afterwards - the core message was clear: Something - a substance, a product - is gravy simply because we say "IT'S GRAVY".
.png)
Then word clouds, bar charts, and pie charts were generated to visualize the distribution of gravy products and item types by supplier, the breakdown of gravy products that are liquid vs. solid - and more. Making the data more accessible and engaging.
.png)
.png)
.png)
Gravy at Thanksgiving - A data story.
It was starting to become obvious from the data that gravy was a lot more than Thanksgiving side dish. In fact, we started to see a trend showing that gravy was often NOT something you'd ever serve at thanksgiving. And we asked for some visuals on it... but at first, ChatGPT got it wrong. (Tell us if we're crazy... but would YOU serve dog food at Thanksgiving!?)
.png)
.png)
So, we tried again. Using the typical "consumer(s)" of the product as a way to more accurately categorize if a product can and should be served at Thanksgiving.
.png)
New product development: 'Universal Gravy'
Now we started thinking.... how can we use this data to envision a BRAND NEW type of gravy product? The concept of "Universal Gravy" was introduced, a gravy product for all of these consumers (including pets?). The final result?: Universal Gravy: Bringing Family and Pets Together.
PRODUCT DETAILS
UPC: 99999999999 Product Name: UNIVERSAL_GRAVY Supplier: Global Flavors Inc. Item Ingredients List: beef stock, chicken stock, onion, garlic, soy sauce, spices Item Type: Gravy Physical Form: Liquid Serve at Thanksgiving: Family Number of Ingredients: 6
.png)
.png)
Gravy, but make it... healthy?
Then we wanted to know... which gravy product is the most health for our families this Thanksgiving holiday? If you're worried about that kind of thing. As it turns out: Fancy Feast cat food is your best bet. We'll skip it, but it was still interesting to look at the comparison of two of the leading human gravy products.
.jpg)
The future of gravy
And, finally, we dove into the potential future of gravy. Based on what we know about gravy today - what could the gravy product and how they exist in the future look like?
Can we make gravy in a more sustainable and less expensive way? And, in the wake of climate change, will how we look at the consistency of gravy change as the world warms? Is this is ridiculous thought, or is there some validity to it? In a consistently warmer climate, could most solid foods become liquid, effectively turning the entire food ecosystem into gravy? This intriguing thought raises questions about how climate change might reshape how we think about and handle goods.
In asking ChatGPT to come with some future concepts for how these gravy products might exist in our future world, we got come fascinating concepts and visuals.
.png)
.png)
While we went down several interesting explorations on gravy as a team, one thing was clear: What began as a simple question about gravy evolved into a captivating journey through why we identify products and classify products certain ways. The multitude of information that can be derived from product data if you ask the right questions.
So.... what should we ask next?
The Latest in CPG Regulations: June 2023
Let’s face it – CPG regulations can be convoluted. They are often (necessarily) filled with complexity and nuance that don’t make it easy to decipher how exactly they may affect you and the products you sell.
We know how important it is for you to stay up-to-date with anything that could impact your business. That's why we're here to fill you in on some interesting updates that we're keeping a close eye on. These recent updates cover four main areas: DOT PHMSA International Harmonization (HM-215Q), Vermont HB 67, and Washington SB 5144, and MOCRA. So, let's dive in.
DOT PHMSA International Harmonization (HM-215Q)
Get ready for some positive changes: The Department of Transportation's Pipeline and Hazardous Materials Safety Administration (PHMSA) has proposed a rule (HM-215Q) that will make your life easier. The new rule was proposed in May, and is likely to be finalized by EOY. Here's what you need to know:
- Button cell batteries: Good news. You'll still need to test button cell batteries installed in equipment, but you won't have to worry about sharing the Test Summary (TS) report anymore. It's a small change, but it means less paperwork and more streamlined compliance for you.
- Lithium battery markings: Say goodbye to the phone number requirement on the lithium battery mark. This simplification will make labeling lithium batteries easier and removes confusion from the supply chain.
- PSN and ID8000 updates: There are a few minor updates to the Proper Shipping Name (PSN) and ID8000. These updates will help you properly identify and ship hazardous materials. It's essential to stay informed about these changes to ensure you're on the right track when it comes to shipping regulations.
Vermont HB 67
Vermont has some interesting legislation in the works, and it's something you'll want to pay attention to. The legislation was delivered on May 12th and comes with 2025 implementation dates. Here's the lowdown:
- Funding the HHW program: The new law would require manufacturers and brands to step up and fund the Household Hazardous Waste (HHW) program in Vermont.
- Increased responsibility on the consumer for haz-waste vs. non haz-waste: This change could mean that the distinction between Haz-waste and Non-haz-waste would be as important to consumers, waste management entities, recyclers, and brands as it is to Retailers.
- Waste handling fees and brand responsibility: The legislation allows for waste handling fees, which will be redistributed to brands, and is typically based on their market share. Orphaned products (products without a brand owner) would be collectively covered by participating brands.
- Exemptions and special considerations: The law includes exemptions for certain products like pesticides, cosmetics, drugs, certain paints, and already covered electronics and batteries. Make sure you're aware of these exemptions to avoid any compliance headaches.
Washington SB 5144
Washington has also got some new regulations coming your way. The legislation was signed on May 11th by the governor, with 2027 implementation dates. Here's the scoop:
- Battery stewardship plan: If you are a “producer”of covered batteries or products containing them, you'll need to participate in a state-approved battery stewardship plan. It's all about responsible management and ensuring proper recycling and disposal.
- Who's a producer? The law broadly defines a producer as battery manufacturers, retail brands, third-party brands, licensees of a brand, importers, or anyone selling the product in the state. It's an inclusive, hierarchical definition to ensure accountability throughout the battery supply chain.
- Battery markings and compliance: although coming into force at a later date, “producers” shall supply, and retailers must collect, a certification that the covered batteries have the required "producer" marking along with the battery chemistry. Lawmakers are placing primary responsibility on the producer, but also require due diligence from retailers.
FDA's Modernization of Cosmetics Regulation Act of 2022
Although published at the end of 2022, there’s significant regulatory change coming that impacts the cosmetics industry:
- FDA regulations for fragrance allergen rules: The FDA is mandated to promulgate regulations for allergen rules within 18 months of December 29, 2023. Keep an eye out for these regulations to ensure your products comply with the new requirements.
- Fragrance allergen ingredient disclosure: Once the list of fragrance allergens is finalized, brands and manufacturers must disclose these allergens on the cosmetic product label . This means you may see changes in product labeling and information provided to customers.
- Does label disclosure equal FDA disclosure? While label disclosure is an important part of complying with FDA regulations, producers of cosmetic products must also register with the FDA and submit a Cosmetic Product Listing, which includes “...a list of ingredients in the cosmetic product, including any fragrances, flavors, or colors, with each ingredient identified by the name, as required under section 701.3 of title 21, Code of Federal Regulations (or any successor regulations), or by the common or usual name of the ingredient”. Will this disclosure of information to the FDA be the same information required on the product label? The answer will impact the logistics of information transparency in the supply chain.
Staying informed about regulatory updates doesn't have to be overwhelming. These updates are here to make your life easier. So stay in the loop, adapt your processes as needed, and reach out to our team of experts with questions at any time, and check back next month for more updates.
Demystifying 1,4 Dioxane: What you need to know
Lately, the regulatory spotlight has been shining on one particularly confusing chemical in consumer products: 1,4-dioxane.
Both California and New York have already issued new guidance targeting 1,4 dioxane. These new restrictions will mean that both retailers and suppliers need to pay attention to where the substance may be, and be ready to confirm that it is not in their products at a certain level. So, we’ve put together a quick look at the reasons behind the urgency around 1,4 dioxane and some tips for how to stay ahead of the curve.
1,4 dioxane: What is it? And why does it matter?
1,4-dioxane is a substance that can be created when making detergents, soaps, and creams, and it’s now considered an environmental contaminant and a probable human carcinogen. 1,4-dioxane contamination can occur rather easily – as a byproduct of the manufacturing processes when making these products. It is also intentionally used at higher concentrations as a solvent in industrial manufacturing processes. Studies have also shown that the harmful substance can easily dissolve in water, which means it could be found in unsafe amounts in drinking water.
All of this has, naturally, made consumers and regulatory bodies worried about finding it in things they use and its possible effects on our health. However a risk evaluation published by the US EPA in 2020 found no unreasonable risks to consumers or bystanders from any conditions of use, including eight consumer uses of surface cleaners, laundry/dishwashing detergents, and paint/floor lacquer where 1,4-dioxane is present as a byproduct.
Regulatory changes and how to keep up
While there is still a lot of unknown around the future of 1,4 dioxane, specific new regulations have started to gain traction. The Food and Drug Administration had previously encouraged suppliers to minimize 1,4-dioxane content, but New York has now taken a proactive approach by implementing a restriction on dioxane contamination levels in products. As of December 31, 2022, cleaning and personal care products are limited to 2 parts per million (ppm), while cosmetics are limited to 10 ppm. And the cleaning and personal care limit was set to reduce to 1 ppm by December 31, 2023.
1,4-dioxane has also been getting attention because of the California Cleaning Product Right-To-Know Act of 2017. Under this new regulation, it must be disclosed on a products website as a “nonfunctional constituent” when it's present at or above 10 parts per million (ppm). Since this chemical is also a carcinogen on the California Proposition 65 list, it might be subject to labeling requirements even below this threshold.
There a few simple - but crucial - steps both suppliers and retailers can take to stay ahead of these changes:
- Brush up on which product categories are affected by these bans. You can find product categories that are likely to fall under the ban on the NY State Department of Conservation website.
- Suppliers: Be ready with specific evidence proving the dioxane content for your products is below the allowable threshold. If a product uses a “ethoxylated” ingredient (commonly employed in the production of personal care, household care products) it might contain a regulated amount of 1,4 dioxane. Determine which products you have contain these ingredients, so you can narrow down which items you need to obtain evidence for and make sure you have that on-hand.
- Retailers: Maintain an open-line of communication with suppliers. Your suppliers know their products’ best and have the information you need to prove that the dioxane levels are below allowable thresholds.
The NY Dioxane Ban, along with evolving regulatory guidelines, can be confusing to navigate and leave more questions than answers. Lean on your suppliers, retailers and regulatory partners to help translate and prepare for these new guidelines as effectively as possible.
By staying informed and maintaining close collaboration with your partners, you can successfully navigate these regulations, avoid costly fines or product delays, and provide consumers with safe and transparent choices.
Ready to join the Product Intelligence™ revolution?
See for yourself how SmarterX can help you with retail compliance, product insights, and building better supply chains.
