Narwhals: enabling universal dataframe support

🔊 Recorded at PyData Berlin 2025,
https://2025.pycon.de/program/JKEHMH/

🎓 How Narwhals enables seamless dataframe interoperability across Pandas, Polars, PyArrow, DuckDB & more—with zero dependencies.

Speakers:
Marco Gorelli

Description:
Narwhals is revolutionizing the Python data science ecosystem by providing a lightweight compatibility layer between dataframe libraries. Marco Gorelli explains how this zero-dependency library enables tool builders to write code once that works seamlessly across Pandas, Polars, PyArrow, DuckDB, PySpark, and more. With 30 million monthly downloads and adoption by major libraries like Altair, Plotly, Bokeh, and Marimo, Narwhals is solving the fragmentation problem that has plagued the dataframe ecosystem. The talk covers the technical architecture, including how Narwhals handles lazy vs eager execution, row ordering, indexing challenges, and SQL compatibility. Gorelli also shares valuable insights on community building, static typing benefits, and the philosophy behind creating a sustainable open-source project. Learn why converting everything to Pandas is a missed opportunity and how meeting users where they are with their preferred dataframe library leads to better performance and fewer bugs.

⭐️ About PyData Berlin:
Since 2014, PyData Berlin has brought together the international Python and data community.
With monthly meetups and a yearly conference, it connects data engineers, scientists, and tool-builders to share ideas, explore open-source technologies, and drive innovation together.

Follow us:
• LinkedIn:   / pydata-berlin  
• X: https://www.x.com/pydataber

Links:
• Conference website: https://berlin.pydata.org/
• Other sessions: https://berlin.pydata.org/conferences...

The conference is organized by
• NumFOCUS Inc.: http://numfocus.org
• Pioneers Hub gemeinnützige GmbH: http://pioneershub.org


If you enjoyed this session, please like, comment, and subscribe to our channel for more insightful talks and discussions.
Share this video with your network to spread the knowledge!

Hashtags:
#Python #PyConDE #PyData #OpenSource #AI #DataScience #MachineLearning #SoftwareDevelopment #LLMs #Community

Acknowledgements:
Special thanks to all the volunteers and sponsors who made this event possible.

About:
NumFOCUS Inc.
supports open-source scientific computing by providing financial and logistical support to key projects like NumPy and Jupyter, promoting sustainable development and collaboration.

Pioneers Hub gemeinnützige GmbH:
is a non-profit fostering innovation in AI and tech by connecting experts and promoting knowledge exchange through events and collaborative initiatives.
www.pydata.org

PyData is an educational program of NumFOCUS, a 501(c)3 non-profit organization in the United States. PyData provides a forum for the international community of users and developers of data analysis tools to share ideas and learn from each other. The global PyData network promotes discussion of best practices, new approaches, and emerging technologies for data management, processing, analytics, and visualization. PyData communities approach data science using many languages, including (but not limited to) Python, Julia, and R.