Details
-
Type:
Bug
-
Status: Resolved
-
Priority:
Critical
-
Resolution: Fixed
-
Affects Version/s: 0.2, 0.3
-
Fix Version/s: 0.3
-
Component/s: Interpreter, RSC
-
Labels:None
Description
in python notebook, using table magic with large dataset include unicode string.
return Exception.
Test Code
test_map = []
for x in range(0, 100, 1) :
test_list = []
for y in range(0, 100, 1) :
test_list.append(x)
test_list.append(y)
test_list.append(u"\u263A")
test_map.append(test_list)
%table test_map
Livy Server Exception
io.netty.handler.codec.DecoderException: com.cloudera.livy.shaded.kryo.kryo.KryoException: Buffer underflow.
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:280)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:149)
at io.netty.handler.codec.ByteToMessageCodec.channelRead(ByteToMessageCodec.java:108)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:787)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:130)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.cloudera.livy.shaded.kryo.kryo.KryoException: Buffer underflow.
at com.cloudera.livy.shaded.kryo.kryo.io.Input.require(Input.java:164)
at com.cloudera.livy.shaded.kryo.kryo.io.Input.readUtf8_slow(Input.java:561)
at com.cloudera.livy.shaded.kryo.kryo.io.Input.readUtf8(Input.java:535)
at com.cloudera.livy.shaded.kryo.kryo.io.Input.readString(Input.java:465)
at com.cloudera.livy.shaded.kryo.kryo.serializers.DefaultSerializers$StringSerializer.read(DefaultSerializers.java:171)
at com.cloudera.livy.shaded.kryo.kryo.serializers.DefaultSerializers$StringSerializer.read(DefaultSerializers.java:160)
at com.cloudera.livy.shaded.kryo.kryo.Kryo.readClassAndObject(Kryo.java:776)
at com.cloudera.livy.client.common.Serializer.deserialize(Serializer.java:63)
at com.cloudera.livy.rsc.rpc.KryoMessageCodec.decode(KryoMessageCodec.java:77)
at io.netty.handler.codec.ByteToMessageCodec$1.decode(ByteToMessageCodec.java:42)
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:249)
tested By master branch, spark 1.6.1