Polars Cloud and Distributed Polars now available

https://news.ycombinator.com/rss Hits: 10
Summary

After working hard since our Polars Cloud announcement last February, we are pleased to officially launch Polars Cloud. Polars Cloud is now Generally Available on AWS. Beyond that, we also launched our novel Distributed Engine in Open Beta on Polars Cloud. You can immediately get started at https://cloud.pola.rs/. After that you can fire a remote distributed query: import polars_cloud as pc import polars as pl from datetime import date with pc.ComputeContext( workspace="<my-workspace>", instance_type="m6i.large", cluster_size=8, storage=64, ) as ctx: in_progress = ( pl.scan_parquet("s3://polars-cloud-samples-us-east-2-prd/pdsh/sf100/lineitem/", storage_options={ "aws_request_payer": "true", }) .filter(pl.col("l_shipdate") <= date(1998, 9, 2)) .group_by("l_returnflag", "l_linestatus") .agg( count_order=pl.len() ) .remote(ctx) .distributed() .execute() ) print(in_progress.await_result().head) Closing the DataFrame scale gap The General Availability of Polars Cloud on AWS marks a major milestone in closing the DataFrame scale gap—the historic divide between the ease of pandas locally and the scalability of PySpark remotely. By making Polars Cloud broadly accessible, we bring to life our mission of delivering fast, flexible and open-source data tools that run everywhere, giving users a single API that seamlessly scales from a laptop to the cloud. Equally significant is the Open Beta of our Distributed Engine, which leverages Polars’ novel streaming architecture to offer not just horizontal but also vertical and diagonal scaling strategies. This design directly addresses the cost, complexity and performance tradeoffs users face today, while making high-performance compute broadly accessible. Together, these launches represent a step-change: remote execution that feels native, distribution without friction, and an architecture built to meet the future of large-scale data processing head-on. 1. What is Polars Cloud Polars Cloud is a managed data platform that enables you t...

First seen: 2025-09-04 10:00

Last seen: 2025-09-04 19:02