[Core feature] Polar dataframe transformer

[pingsutw]

Motivation: Why do you think this is important?

Similar to https://github.com/flyteorg/flytekit/tree/master/plugins/flytekit-polars, we need a polar transformer in the v2 SDK.

Goal: What should the final outcome look like, ideally?

Have a new polar plugin in the flyte-sdk

Describe alternatives you've considered

NA

Propose: Link/Inline OR Additional context

No response

Are you sure this issue hasn't been raised already?

• Yes

Have you read the Code of Conduct?

• Yes

[runllm]

The Flyte v1 SDK already provides a Polars DataFrame plugin (flytekitplugins-polars) that supports polars.DataFrame as a data type with StructuredDataset, including encoding/decoding to Parquet. However, there is no explicit documentation or release indicating a Polars transformer or plugin for the v2 SDK as of now. The v2 SDK supports DataFrame interoperability via flyte.io.DataFrame, but native Polars support in v2 is not mentioned in the available sources.

If you need Polars support in v2, you would need to implement a new plugin or type transformer, similar to the v1 approach, and register it with the v2 DataFrameTransformerEngine. There is no out-of-the-box Polars plugin for Flyte v2 SDK at this time. Would you like more detail on how to implement such a transformer for v2?

Sources:

• flytekit-polars plugin README
• Polars plugin implementation (v1)

---

This is an AI-generated response and your feedback is appreciated! Please leave a 👍 if this is helpful and 👎 if it is not.

Mention @RunLLM to ask follow-up questions. See runllm.com for more!