The future of the chemicals industry: Fostering innovation through collaboration

The global demand for the next wave of innovation in the chemicals sector is growing. Successful market introduction of these innovations requires collaboration between established companies, startups, and investors.

Dr Francesco Dergano
12 min readApr 14, 2024

The chemicals industry stands at a crucial juncture, needing radical change to address the swiftly evolving demands of global communities and to capitalize on rapid technological advancements. This sector is called upon to innovate significantly, with potential developments ranging from plant-based chemicals and biodegradable polymers to advanced battery materials and ionic liquids.

The industry has already made notable advancements in areas such as lithium batteries, metallocene linear low-density polyethylene (LLDPE), and organic light-emitting diode (OLED) displays. With a robust ecosystem of established companies, startups, and venture capitalists now in place, there is a prime opportunity for further innovation to tackle global issues.

To better understand the barriers to innovation within the chemicals industry, we conducted a survey of over 200 senior leaders from North American chemical companies, which included executives from leading firms, investment entities, and startups. Our investigation focused on the primary challenges faced by the industry, obstacles to collaboration, industry responses to disruptions, ongoing initiatives, and potential drivers for scaling innovation.

Our findings underscored a critical need for enhanced collaboration within the industry to foster a dynamic innovation ecosystem. Highlights from the survey revealed that:

  • Innovation in the chemicals sector spans the entire value chain, with contributions from both industry leaders and startups.
  • Chemical process innovation emerged as the most significant challenge, as acknowledged by a broad range of survey participants.
  • While both industry veterans and startups encounter similar hurdles in scaling innovations, each possesses unique strengths that could be leveraged to offset the other’s shortcomings. However, barriers that hinder effective collaboration still exist.

In our analysis, we outline strategies for industry stalwarts, newcomers, and investors to engage in key activities that can break down these barriers, enhancing collaboration and nurturing an environment conducive to innovation. This collaborative effort is essential for the industry to play a pivotal role in the forthcoming materials revolution. We also discuss specific actions these parties can take to cultivate this ecosystem while simultaneously increasing their own value.

Disruptive trends challenge the chemicals industry

The chemicals industry is confronting a wave of disruptive trends. Customers are increasingly demanding more sustainable practices and products, such as the substitution of traditional synthetic materials like detergents and polymers with eco-friendlier alternatives. Simultaneously, global decarbonization efforts are intensifying, with stringent environmental regulations driven by government carbon-reduction initiatives necessitating a significant reduction in emissions – approximately 60% by 2030 – to align with the 1.5° global warming pathway.

Since 2020, the industry has also faced soaring inflation and significant supply chain disruptions, posing unique challenges, especially for global, energy-intensive operations. These conditions coincide with an opportune moment for transforming chemical manufacturing processes through digital technology to enhance cost efficiency. Moreover, the shift towards e-commerce has altered customer interactions and expectations, further emphasizing the need for digital adaptation.

Despite these pressures, the sector has seen little innovation that is discovery-led, with all of the top ten chemical products having been developed over half a century ago. However, change is on the horizon. Recent surveys show increasing investments from both established industry leaders and visionary investors. Chemical startups are recognizing an opportunity to introduce novel chemistries to the market, although they frequently underestimate the necessary time, cost, and effort for successful market entry and scaling.

There’s a clear need for more collaboration within the industry. Yet, as our findings suggest, there’s been a reluctance to collaborate effectively, with many viewing potential partners as competitors. Embracing collaboration could be key to unlocking the transformative potential of these disruptive trends in the chemicals industry.

Three big challenges to innovation

Our research, corroborated by survey results, identifies three significant barriers to innovation. It outlines eleven innovation themes across five key stages of the value chain: discovery, production, delivery, application, and management (as shown in Exhibit 1).

Eleven innovation themes are pivotal in driving business development within the chemicals industry.

Exhibit 1:

Discovery

  • Use of Al and advanced analytics to X predict and uncover new chemical molecules or combinations that provide superior pertormance to solve novel problems that traditional chemicals have not (eg, Al-enabled protein discovery for pharmaceuticals)

Production

  • New chemical production systems that are capital light, decentralized, multi-use, and/or can be located closer to the point of use, resulting in desirable unit economics, higher ROIC, and minimized inventory/transport
  • Processes to reduce emissions in X the chemical production process (eg, electrification, new reactor design, non-virgin feedstock, etc) and novel manufacturing to create commodity chemicals in an ecofriendly, modular manner to enable high process value chains (eg, lithium extraction from brine resources)
  • Production of chemicals and fuels using CO2/carbon as feedstock (in combination with hydrogen)
  • Use of biomass and other biomaterials as inputs in the production of chemicals, energy, and materials vs traditional petrochemical inputs and bioprocesses (eg, fermentation, biochemical process with enzyme, etc)

Delivery

  • Digital marketplace or platform that allows digital interactions and transactions across chemical value chain players (eg, buyers, sellers, distribution) on sourcing, spec / material safety data sheets management, inventory, etc
  • Tools used to enable transparency into the supply chain and manufacturing process of chemicals (eg, SaaS solutions, raw material intelligence)

Application

  • Individual materials and/or material system engineering that support sustainability initiatives such as the transition to renewable sources of energy and electrification (eg, EV production, insulation, flame retardants, coatings for wind turbines)
  • Individual materials and/or material system engineering that support the development of new industries and innovations (eg, space travel, medical grade plastics)

Management

  • Reduce plastic waste/leakage and X support plastic recycling and reuse (eg, waste to plastic, bioplastics, pyrolysis recycling, mechanical recycling etc) and/or reconfiguration of plastics value chain to support circularity (eg, product trade-in system)
  • Reduce waste/leakage and support recycling and reuse of other hazardous and non-hazardous chemical products (eg, biofuel, reretining lubricant, tires to mulch) and/or reconfiguration of chemical value chain to support circularity (eg, redesign of products for easier repair)

Chemical process innovation and AI-assisted chemical discovery are the two most important themes

Our survey reveals that both industry leaders and startups, along with investors, concur that chemical process innovation and AI-assisted chemical discovery are the two primary themes. These are seen as the most challenging yet crucial areas, believed to offer the greatest potential for near-term growth (as detailed in Exhibit 2).

Themes vary among respondents in the perception of difficulty and importance

Exhibit 2: 1 “Taken from question, “Which innovation themes do you think will be the most challenging for the current chemical industry to respond to?” 2 Taken from question for incumbents, “Select the top three innovation themes that are most important to your company’s growth,” and the question for start-ups, “Select the top innovation theme(s) your company focuses on.”Source: Innovation survey in the chemicals sector 2022 (North America), McKinsey, 2022

Yet, opinions diverge on the next most challenging theme. Investors and startups see the conversion of CO2 to chemicals as the third most challenging theme, whereas industry leaders focus more on the development of new applications for specialty materials aimed at sustainability (like electric vehicle production or insulation) and other sectors (including space travel or medical-grade plastics).The conversion of CO2 to chemicals is generally seen as more challenging than it is crucial for growth, whereas AI-assisted discovery is viewed as slightly more critical to growth than it is challenging.

Significant interest to invest in and scale innovations in the industry

There is considerable interest from both industry leaders and investors to fund and expand innovations they believe are crucial for growth. Over 70 percent of survey respondents plan to invest an average of more than $50 million in various innovation themes over the next five years (as shown in Exhibit 3).

Industry leaders and investors are planning significant investment across innovation themes, with industry leaders being more bullish

Planned investment in the next two years by investment theme,’% of respondents

Exhibit 3: ‘Taken from question, “How much does your company plan to invest for each of the below innovation themes?” Source: Innovation survey in the chemicals sector 2022 (North America), McKinsey, 2022

Overall, industry leaders are more eager to invest across various themes compared to investors: 94 percent of industry leaders versus 71 percent of investors plan to invest more than $50 million. Investors are primarily interested in themes related to discovery and production, while industry leaders concentrate their investments on themes related to delivery and application.

Industry leaders are prepared to place larger bets, with 35 percent planning to invest more than $500 million in areas such as digital platforms and e-commerce (75 percent), biofeedstock and bioprocesses, and specialty materials for new applications. On the other hand, investors are more cautious, with only 10 percent planning to invest over $500 million, showing the highest interest in specialty materials for new applications, the circularity of chemicals, and chemical traceability.

Investment preferences align with the perceived importance of each theme:

  • Chemical process innovation is considered the most critical theme, with 90 percent of investors and 97 percent of industry leaders planning substantial investments exceeding $50 million.
  • AI-assisted discovery, though not ranked as the most important theme, is still a focus for significant investment, with 43 percent of investors and 75 percent of industry leaders planning to invest over $100 million.
  • Decentralized chemical production ranks fourth in importance among investors, with 69 percent planning to invest between $100 million and $500 million. Although industry leaders rank it ninth in importance, they view it as the third most challenging theme, with 89 percent planning to invest more than $100 million, and 33 percent considering investments above $500 million.
  • Circularity of chemicals shows the most varied interest – despite its lower ranking, all investors plan to invest less than $50 million, while 88 percent of industry leaders plan to invest more than $50 million over the next two years.

Given the complexity of the chemicals industry ecosystem, the actual investments required are likely to exceed initial estimates, emphasizing the need for collaboration between industry leaders and investors to foster innovation and scale these initiatives effectively.

Industry leaders and start-ups have complementary strengths to balance the weaknesses

Industry leaders and startups bring complementary strengths to the table, which can help offset each other’s weaknesses. Industry leaders, with their robust R&D capabilities, access to capital, and established go-to-market resources, are well-equipped to assist startups in scaling up breakthrough technologies. They can also provide startups with access to customers and help with product validation, which enhances the startups’ credibility and facilitates their growth.

Both groups, however, encounter challenges in scaling innovations. According to our survey, common obstacles include resource allocation, organizational structure, and operating models. Industry leaders often face issues with designing solutions that are fit for purpose, gaining full support from their C-suites, and acquiring necessary expertise. Startups, on the other hand, typically struggle with securing feedstock, accessing funding, and navigating complex value chains. Additionally, startups may not fully appreciate the difficulty and time required to scale up and commercialize innovations, despite early successes in the lab (as illustrated in Exhibit 4).

Exhibit 4: ‘Q: What are the biggest barriers your organization is facing in scaling your top innovation theme(s)?

Industry leaders who participated in our survey highlighted strategy alignment and a robust proof of concept as the top criteria for an ideal partnership with startups. For approximately half of these leaders, the primary advantage of partnering with startups is the leverage it provides for technology development.

In return, startups can help industry leaders gain access to new technologies and accelerate the incubation process with more flexibility. Startups also value these partnerships for the capital support available from industry leaders that aids in scaling operations, as well as their connections to other key players in the industry. Moreover, 63 percent of startup respondents are keen to tap into the industry leaders’ go-to-market strategies and R&D resources.

Collaborative partnerships are increasingly recognized as crucial for growth, with 60 percent of both startups and industry leaders viewing each other as potential collaborators in value chains or R&D, rather than as competitors (as detailed in Exhibit 5).

Incumbents’ and start-ups’ perception of each other,’ %

Exhibit 5: ‘Q: “How do you typically view established chemical players across each innovation theme?” Refers to the percentage of time a response was indicated as a top-three response. Only start-ups that selected the theme as an area of focus and incumbents that selected the theme as a top-three important theme were accounted for. Source: Innovation survey in the chemicals sector 2022 (North America), McKinsey, 2022

Barriers to increasing collaboration: Lack of transparency and trust comes out tops

Several barriers currently prevent startups and industry leaders from enhancing collaboration. A significant issue identified by both parties is the lack of transparency, which they see as a major risk to forming successful partnerships. Establishing clear targets, downside protection, and rules of engagement could mitigate these concerns.

Industry leaders tend to develop innovation themes internally, through in-house R&D or by creating new business units. However, they prefer to develop certain areas like circularity of plastics and e-commerce externally, likely due to insufficient internal capabilities or the need for wider ecosystem collaboration. In their internal innovation efforts, industry leaders face obstacles related to resources, organizational structures, operating models, and technology.

Startups, on the other hand, sometimes adopt a more competitive stance toward industry leaders in areas such as digital platforms, e-commerce, CO2 to chemicals, and specialty materials for sustainability applications. They also face challenges in scaling their operations, including acquiring sufficient capital, feedstock, and organizational capabilities, and navigating the value chain efficiently.

In terms of perceived risks, 22 percent of industry leaders cite technology failure, and 15 percent cite transparency issues. Among startups, 21 percent view the lack of speed and transparency as risks, and 15 percent are concerned about a lack of technological expertise. To reduce risks in partnerships, both startups and industry leaders must focus on increasing transparency and improving alignment (as illustrated in Exhibit 6).

Lack of transparency is a key risk for both incumbents and start-ups

Exhibit 6: ‘Q: What are the most important criteria when determining whether to partner with a start-up/incumbent? What do you consider the biggest risk factors when partnering with start-ups/incumbents? Source: Innovation survey in the chemicals sector 2022 (North America), McKinsey, 2022

Key actions to cultivate an innovative economic ecosystem

The challenge of fostering an innovative economic ecosystem may seem formidable, but we have pinpointed several key actions that can help create a more cooperative and dynamic environment for innovation in the chemicals industry going forward.

The Imperative for Change: Collaborative Opportunities Abound

Despite existing barriers, many industry leaders and startups recognize the advantages of collaboration, presenting abundant opportunities for partnerships. Breaking down silos sooner rather than later is crucial. Traditional venture investors often find the capital and time requirements for successfully commercializing technology challenging, but these gaps can be bridged through collaboration. Established companies bring expertise and experience in large-scale capital deployment and market expansion. Some successful examples of such collaborative efforts are already visible in the plastic waste sector, where partnerships between recycling startups and established companies are fostering much-needed circularity.

Both startups and industry leaders exhibit a mutual interest in themes like chemical process innovation and AI-assisted discovery, showing readiness to collaborate. While startups may adopt a somewhat competitive stance towards industry leaders, they increasingly view each other more as potential partners than competitors.

Strategic Steps for Collaboration, Innovation, and Value Growth in the Chemicals Industry:

  • Industry Leaders: Key actions include signaling strategic interests and making corresponding investments, both at early stages (venture investments) and later stages (acquisitions). This helps startups prioritize their innovation roadmaps effectively, potentially creating significant value. Direct investments in or acquisitions of startups at the commercialization stage can provide the necessary talent, expertise, and resources to scale innovations successfully.
  • Startups: Should aim to understand the needs and priorities of industry leaders and focus on areas that can significantly impact these incumbents. By setting realistic, achievable goals for the near to mid-term and using their inherent agility, startups can expedite reaching crucial commercialization milestones, enhancing their chances of a successful exit.
  • Investors: Can serve as crucial connectors by acting as independent, credible advisors, bridging the gap between startups and industry leaders to facilitate collaboration. This role has been effectively demonstrated in the aviation industry, where investors have brokered deals that combine investor capital, startup technology, and industry leader resources to develop and commercialize innovations like biofuel.

These collaborative efforts are not just beneficial but necessary for driving innovation and growth in the chemicals industry.

In Conclusion

The chemicals industry is pivotal in tackling global energy and sustainability issues but must address its innovation shortfall. Traditionally, industry leaders, investors, and startups have viewed each other more as competitors than collaborators, hindering joint efforts. Fortunately, there is significant agreement among these key players about the critical challenges facing innovation in the sector. By taking decisive steps to overcome these obstacles, the chemicals industry can resolve its innovation challenges.

Now is the moment for the chemicals sector to seize these opportunities, overcome its fragmentation, and begin collaborating to unlock growth potential and contribute significantly to the global sustainability effort.

--

--

Dr Francesco Dergano

CEO of Skydatasol —Managing Principal of Kamiweb Project —Lead Research Manager and CISO of The National Security Framework—Full-Time Student in London