在突触笔记本中将表从火花池转换为熊猫时出错

问题描述

我在火花池中有一张桌子，我可以在突触中访问它。但是我不能称它为熊猫。以下是错误：

%%pyspark
rawdata = df.select("*").toPandas()

ArrowInvalid : Casting from timestamp[us,tz=Etc/UTC] to timestamp[ns] would result in out of bounds timestamp: 33460473600000000
Traceback (most recent call last):
  File "/opt/spark/python/lib/pyspark.zip/pyspark/sql/dataframe.py",line 2132,in toPandas
    pdf = table.to_pandas()
  File "pyarrow/array.pxi",line 715,in pyarrow.lib._PandasConvertible.to_pandas
  File "pyarrow/table.pxi",line 1565,in pyarrow.lib.Table._to_pandas
  File "/home/trusted-service-user/cluster-env/env/lib/python3.6/site-packages/pyarrow/pandas_compat.py",line 779,in table_to_blockmanager
    blocks = _table_to_blocks(options,table,categories,ext_columns_dtypes)
  File "/home/trusted-service-user/cluster-env/env/lib/python3.6/site-packages/pyarrow/pandas_compat.py",line 1115,in _table_to_blocks
    list(extension_columns.keys()))
  File "pyarrow/table.pxi",line 1028,in pyarrow.lib.table_to_blocks
  File "pyarrow/error.pxi",line 84,in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: Casting from timestamp[us,tz=Etc/UTC] to timestamp[ns] would result in out of bounds timestamp: 33460473600000000

解决方法

暂无找到可以解决该程序问题的有效方法，小编努力寻找整理中！

如果你已经找到好的解决方法，欢迎将解决方案带上本链接一起发送给小编。

小编邮箱:dio#foxmail.com (将#修改为@）

azure-synapse pandas pandas