HackDefend Labs

Sysinternals case writeup

2022-11-25T00:00:00+00:00

This is a small writeup for the sysinternals case recently published by Ali Hadi.

The case starts after donwloading a sysinternals tool that is suspected to be a malware since it didn’t behave as expected and the user noticed a slowness in the system. To start the case analysis, file system and evidence of execution artifacts are the first two traces to start analyzing. MFT is a good candidate for file creation and amcache or prefetch for evidence of execution.

MFT showed that two files with the name sysinternals.exe were created, first file was clean and is likely a currpted since it doesn’t contain MZ header. And second file was malicious and resulted in 32 hits on VirusTotal.

File Path	Hash
Users\Public\Downloads\sysinternals.exe	EE18B3A542E2C27AB8E7506BC4B39379
Users\IEUser\AppData\Local\Packages\Microsoft.MicrosoftEdge_8wekyb3d8bbwe\AC#!001\MicrosoftEdge\Cache\WMFWC1O7\sysinternals[1].exe	D1A27B871A86C5371215F71885862CFF

Looking at the sysinternals[1].exe using PEStudio shows the use of windows functions such as URLDownloadToFileA, InternetOpenUrlA and ShellExecuteA, which is a good indcation to look for another sample possibiley downloaded form internet.

Following are Intersting strings inside sysinternals[1].exe

/C c:\Windows\vmtoolsIO.exe -install && net start VMwareIOHelperService && sc config VMwareIOHelperService start= auto
c:\Windows\Temp\Hex2Dec.zip (there is no evidenec that the file was created on disk)
c:\Windows\vmtoolsIO.exe

Strings and code indicate the existence of other files, as well as installing a malicious service called VMwareIOHelperService.

Next step is to look for network communication to know from where those files were downloaded. Using a sandboxing solutions is quick win to study the malware behaviour as well as to get a copy of PCAP. Fortunately the sample was available in Virustotal with the malware behaviour report. Virusttotal shows many maliciuos TTPs. Hosts file modification was interesting. Disk image hosts file was indeed modified.

Since the domain was locally hosted, it won’t be possible to setup the same server. Luckily the file vmtoolsIO.exe is still avaiable at c:\Windows.

From the strings of sysinternals[1].exe we can run the vmtoolsio.exe with commandline -install to cause the service installation. Once

Vmtoolsio.exe installs a service that deletes all prefetch file and keep deleting any newly created prefetch files.

Prefetch files were deleted as a result of the service created out of vmtoolsio.exe, however we can know the first execution from MFT creation time of the .pf files when first executed.

A tour inside a SOC analyst mind

2022-09-09T00:00:00+00:00

As a mentor in a SOC, I often receive questions from junior analysts about the best way to approach analysis. To help them develop their skills, I sometimes ask them to come up with their own analysis before I offer my support. It’s always interesting to hear their different conclusions and ways of thinking. Some analysts have strong analytical abilities, while others need to be aware of human thinking limitations and biases and how to avoid them to some extent.

SOC jobs are focused on analysis and require the application of decision-making skills that are impacted by the maturity of an individual’s cognitive abilities. In this blog post, I will attempt to list some of the analysis pitfalls that I have observed during my interactions with analysts in a SOC.

Typically, a SOC will have a dashboard that contains a list of security alerts. In the first 10 seconds after looking at the dashboard, a complex thought process occurs in the analyst’s mind to select the right alert for investigation. Like any human, this thought process is likely to have pitfalls. In general, good analysts have better decision-making skills and analytical thinking capabilities than average people. They base their moves on the rules of logic. Applying good analytical techniques is crucial for analysis because not all information will be available. Analysts have to go through layers of abstraction to build context and make decisions.

Video about decision making.

The process of selecting the right alert for investigation is a decision-making skill that is influenced by a combination of gut feeling and cybersecurity experience. Knowing the potential pitfalls that an analyst may unknowingly fall into can help to improve their thinking and analysis. To make it easier to understand how SOC analysts work, a typical alert analysis can be divided into three main phases:

1- Alert selection

2- Analysis

3- Security writing

Each phase has its own potential pitfalls, and the quality of the analysis can vary depending on the analyst’s experience and metacognitive awareness. Some examples of alerts that may appear in a SOC detection platform are:

A sudden spike in network traffic from a specific IP address
An unusual number of login attempts from a particular user
A suspicious file detected on a client’s endpoint

Each of these alerts requires a different approach and decision-making process, and the analyst must be aware of the potential pitfalls and biases that may affect their analysis. Following is a list of alerts in a typical SOC dashboard, analysts would make a guess which alert is worth investigating and what to be ignored and this is where many mistakes are done.

Alerts
T1059.001 - Powershell cmdline obfuscation
Internal Port Scan
T1003.001 - ProcDump64.exe execution
User agent Log4j signature
Log4j Scanning
Log4j Scanning
Log4j Scanning
Log4j Scanning
Log4j Scanning
Log4j Scanning
Log4j Scanning
Log4j Scanning
Repeated IPS hits for same IP Address
Malicious file detected by AV
T1053 - Scheduled Task/Job installation

Following is a simple graph showing the possible mistakes an analyst might do during alert analysis.

Alert Selection

Alert selection is crucial and I always ask analysts, How they select the alerts and why. I always try to improve the level of understanding of the rules based on the careful observation of how alerts are selected.

Alert familiarity

Alerts should be selected based on risk, but analysts often choose alerts based on what they think they know how to analyze. In most cases, alerts are selected based on detection rules or data that the analyst is comfortable with. To reduce this pitfall, the rule on-boarding process should include training on newly deployed rules, and the rule information section should provide clear explanations and resources to help analysts understand the rules. Analysts may avoid selecting alerts for PowerShell obfuscation because they fear that it may be difficult to decode and analyze. Instead, they may choose AV alerts because they are easier to understand and escalate. Analyzing patterns in how alerts are selected can help identify knowledge gaps in the team.

Inductive Reasoning

When multiple alerts coincidentally fire at the same time, analysts may quickly generalize their conclusion about all alerts by only checking one or two. This is due to the static nature of alert titles, which can lead to erroneous conclusions. By being aware of this mistake, analysts can avoid quickly jumping to conclusions and ignoring alerts. From a platform perspective, alert titles should be dynamically derived from the content of logs to help prevent this issue.

Availability Bias

Analysts often use their previous knowledge about the infrastructure or detection rule to make decisions. For example, fixed scanning schedules can cause many rules to fire. Without properly tuning the SIEM to avoid such alerts, analysts may ignore a set of alerts or behaviors, assuming that it is a scan because they are used to seeing these alerts. Improving the tuning of the SIEM can help prevent this issue.

Another example is the regularl uncoordinated pentesting activity, if it’s known that pentester is working on some network without proper communication, alerts could be loosly analyzed due to a previous knowledge of some sort of activity is going on in the network. Applying this to availability bias to above alert list, many log4j scanning alert may be skipped thinking that it’s the scheduled scan.

Analysis

The same though process during alert selection repeated again during the first 10 seconds of the drill down. More difficult time expensive choices an analyst has to select.

Asking the right question

Analyst has to actively think about right question to be asked, knowing why/how the answer will help to ask the next question, and how all questions may lead to answering a hypothesis in mind. Senior analyst will have more hypotheses than junior analysts. The more hypotheses are generated the more questions will be asked. Prioritizing the list of hypotheses and the related question is what is so called good analysis choices.

Chris Sanders research CREATIVE CHOICES: DEVELOPING A THEORY OF DIVERGENCE, CONVERGENCE, AND INTUITION IN SECURITY ANALYSTS tries to understand how SOC analyst generate hypotheses and ask the related investigative questions.

Weak correlation

It’s suffice to say the well known phrase in analysis Correlation does not imply causation. Each move and conclusion should be built on a strong foundation of evidence that are derived by applying rules of logic. Cause-and-effect relationships plays a big role in analysis, experience in IT and and the network being protected.

Trying to prove the impossible

A fundamental question to consider during any investigation is whether we need to prove or disprove a hypothesis. Knowing which option to choose can save a lot of time, and in some cases, proving a hypothesis may be impossible. Security systems are made up of layers of abstraction that can be leveraged to facilitate investigation. Breaking down the situation into smaller areas with relationships and illuminating the relationships between those areas by asking the right questions can help disprove hypotheses or establish a relationship to a particular scenario. For example, in our dashboard, if the logs are not clear about why procdump64.exe is used, an analyst may think that they need to prove that credential dumping occurred in the system and find evidence to support this. However, it may be easier to find evidence about why procdump64.exe was used instead.

Anchoring and confirmation bias

During analysis, it’s very hard to avoid the temptation to go in a new investigation path as new information are available. Analyst For example may think communication over port 4444 in Repeated IPS hits for same IP Address alert is metasploit where it’s not necessarily the case. Avoiding the new investigative paths is important, whenever a new path is to be considered for analysis, the cost of going into the analysis in terms of time, and whether the overall hypothesis put is contributing in solving the case.

Security Writing

Good analyst should have clear thinking that is reflected on writing. The Ability to accurately write about the events timeline crucial. Analysts sometimes make the mistake to write content to prove their sophistication and deep analysis where it should have been an easy to write an actionable instructions only. in addition recommendations must be reasonably selected.

Analyzing SOC Data Using Jupyter Notebook

2022-04-28T00:00:00+00:00

Detection Controls usually focus on telemetry collection, and in most cases lack data analytic capabilities. Hence, It would be necessary to export the data to perform data manipulation and advanced detection analytics. Jupyter Notebook is a great tool for this purpose and has no limits for what you can do with the data. Following are some of the use cases I use Jupyter in Security operations.

Working With Unstructured Data

SOC analysts tasks may involve using tools that produce unstructured to semi-structured data. To be able to analyze the data, values must be structured and parsed to appear exactly like spreadsheets. One way to do is via regex. Patterns will be identified and put in columns.

One example is Loki scanner. The tool generates one file for every endpoint. Parsing and combining all the files into one platform is the goal, especially when scanning hundreds of endpoints.Following snippet parses the rule name, the matched Yara rule and description into pandas columns from files generated with loki –csv options.

cols = ['Hostname','Type', 'File',"Rule Name","Description"]
df = pd.DataFrame(columns = cols)
row = {}

for line in lines:
    try:
        row['Hostname'] = line.split(',')[1]
    except IndexError:
        continue
        
    if "ALERT" in line or "WARNING" in line:
        if "ALERT" in line:
            row['Type'] = "Alert"  
        elif "WARNING" in line:
            row['Type'] = "Warning"
    else:
        row['Type'] = np.NAN
    match = re.search('FILE:\s(.*?)\sSCORE',line)
    if match:
        row['File'] = re.search('FILE:\s(.*?)\sSCORE',line).group(1)
            
    match = re.search('Yara Rule MATCH:\s(.*?)\s',line)   
    if match:
        row['Rule Name']= re.search('Yara Rule MATCH:\s(.*?)\s',line).group(1)
    match = re.search('DESCRIPTION:\s(.*?):',line)
    if match:
        row['Description']= re.search('DESCRIPTION:\s(.*?)REF:',line).group(1)
      
    df=df.append(row, ignore_index=True)

Once the data is represented as columns, all various analysis functions can be applied to understand the data. Complete example of using jupyter notebook with loki files can be found in this notebook.

The same can be applied to working with unparsed SIEM logs. It’s easier to export and perform analytics in Jupyter than parsing the logs in the SIEM.

Detection Rules Tuning

Detection technologies if not continuously tuned will generate high number of alerts that beat all analysts speed and energy. Another use case of Jupyter notebook is fetching and analyzing the noisy alerts data.

Working with APIs is to some extent more flexible than web portals. Qradar for example offers an API interface to work with offenses data.

offenseURL = 'https://IP/api/siem/offenses?filter=start_time%20%3E%20' + str(int(startday)) +"%20and%20start_time%20%3C%20" + str(int(endday))
crURL = 'https://IP/api/siem/offense_closing_reasons'

headers=  {
           'Range': 'items=0-100000','Version': '16.0',
           'Accept': 'application/json',
           'SEC': 'TokenHere'
          }

#offenses
ofR = requests.get(url = offenseURL,verify=False, headers=headers)
data = ofR.json()
offenses = pd.DataFrame(data)
# Closing reason 
crR = requests.get(url = crURL,verify=False, headers=headers)
closeingReasons = crR.json()

Qradar represent time in Unix Epoch time. Additional column called “offense_time” will be added and contains human readable timestamps.

offenses['offense_time'] = pd.to_datetime(offenses['start_time'],unit='ms')
offenses.set_index('offense_time')
offenses['offense_time'][1]

To analyze closing reason of the fired rules, mapping the name of the reason to its ID is required since the offense API doesn’t directly replace the ID with title.

for cr in closeingReasons: 
    offenses.loc[offenses['closing_reason_id'] == cr['id'], 'closing_reason'] = cr['text']
    print("ID: ", cr['id'], "   Name: ", cr['text'])

Analysis could be as simple as visualizing the escalated offenses by day or extracting most noisy offenses evidence by iterating each offense data into a dataframe.

Escalated cases by day

Many other statistics can be extracted once the data is ready. As an example, number of escalated offenses each day.

fig, axs = plt.subplots(figsize=(12, 4))
offenses[offenses['closing_reason'] == 'Escalated'].groupby(offenses['offense_time'].dt.day)['id'].count().plot(kind='bar', rot=0, ax=axs)

Reducing noisy evidence

Sometimes, Offenses require tuning for only one or two evidences to reduce most of the noise. Below code extracts the most noisy evidence causing offenses to fire.

# Top 10 triggered offense 
Names_ofTop10 = offenses["description"].value_counts().head(10).rename_axis('description').reset_index(name='counts')["description"]
top10 = pd.merge(offenses,Names_ofTop10,on='description')

top10_by_group = top10.groupby("description")
for name, group in top10_by_group:
    fig.suptitle(name)
    #print(group["offense_source"].value_counts().head(10))
    plt.figure()
    plot_data = group["offense_source"].value_counts().head(10)
    plot_data.plot(kind="bar",color=color, label=name)
    plt.show()
    print(name)
    print("\n")

One sample output is the following graph. It’s clear that almost all offenses were fired because of one IP and if tuned, the rule FP will be acceptable.

Complete Jupyter Notebook for analyzing Qradar SIEM data can be found Here.

Time Zone Conversion

During IR engagement, data gathered may have different timezones. Using Pandas to plot or create timeline is easy with the help of to_datetime() Pandas function.Many suggest to use Coordinated Universal Time (UTC) for log analysis, but I think you should use a timestamp of most logs gathered. If most of artifacts gathered were from timezone x, then all other should be converted to the same, provided the people reading the report are located in the same timezone, otherwise UTC is preferred.

Logs with no timezone are called timezone naive, and logs with timezone are called timezone aware. First step is to ensure that all logs are timezone aware. Suggested to deal with multiple timezones by putting each timezone in its own dataframe or series.

Example:

Pandas can convert time between timezones using tz_convert() function.

time = pd . Series (['2022-04-22 08:00:00+00:00'])
utc_s = pd.to_datetime( time , utc = True )
utc_s.dt.tz_convert('Asia/Riyadh')

Similarly, tz_convert() can be used to go back to UTC.

local.dt.tz_convert('UTC')

SOC KPI and Reporting

Although, solutions such as PowerBI or SOAR are more suitable for building automated dashboard and generating reports, However Jupyter Notebooks are doing a great job with the help of visualization libraries to do the same.

KPIs are usually generated from multiple sources, especially for MSSPs where the business model deals with different solutions such as EDRs, NDRs, SIEMs …etc. Writing notebooks to automate the process worked fine. The notebook implementation will be written in a way that multiple data sources will be fetched to generate KPIs or generate reports.

One example use case is analysis of escalated incidents by analysts. It’s sometimes not easy to directly plot the data if there is no standard for escalated tickets naming convention. Attaching tags to rows that contain specific words or match a regex can help to unify the data under standardized categories.

dfc.loc[dfc['cases'].str.contains("logon"), 'tag'] = 'Logon' 
dfc.loc[dfc['cases'].str.contains("login"), 'tag'] = 'Logon' 
dfc.loc[dfc['cases'].str.contains("malware"), 'tag'] = 'AV' 
dfc.loc[dfc['cases'].str.contains("defender"), 'tag'] = 'AV' 
dfc.loc[dfc['cases'].str.contains("symantec"), 'tag'] = 'AV' 
dfc.loc[dfc['cases'].str.contains("authentication failure"), 'tag'] = 'Logon' 
dfc.loc[dfc['cases'].str.contains("openssl"), 'tag'] = 'OpenSSL' 
dfc.loc[dfc['cases'].str.contains("injection"), 'tag'] = 'SQLInjection'
dfc.loc[dfc['cases'].str.contains("scan"), 'tag'] = 'PortScan'
dfc.loc[dfc['cases'].str.contains("port"), 'tag'] = 'PortScan'
dfc.loc[dfc['cases'].str.contains("geo"), 'tag'] = 'Logon'
dfc.loc[dfc['cases'].str.contains("waf"), 'tag'] = 'WAF'
dfc.loc[dfc['cases'].str.contains("Intelfeed1"), 'tag'] = 'MaliciousIP'
dfc.loc[dfc['cases'].str.contains("Intelfeed2"), 'tag'] = 'MaliciousIP'
dfc.loc[dfc['cases'].str.contains("suspicious communication"), 'tag'] = 'MaliciousIP'
dfc.loc[dfc['cases'].str.contains("user added"), 'tag'] = 'UserEvents'
dfc.loc[dfc['tag'].isnull() , 'tag'] = 'Other'

Once the data is properly categorized, any aggregation method will be easily applied.

dfc['tag'].value_counts().plot(kind='bar',figsize= (11,6))

Full Notebook example for analyzing and generating KPIs of Qradar SIEM data can be found in this repository.

Advanced Detection Analytics

One interesting example is the implementation of RITA (Real Intelligence Threat Analytics) in Jupyter Notebook for beaconing detection. RITA-J is one of the examples I use to show the possibility of using Jupyter Notebook for advanced security detection use cases.

Remote Full Packet Capture

2021-11-13T00:00:00+00:00

Tool such as TCPDump captures full packet data and store to desk, it doesn’t provide a way to capture and stream the packets to a remote destination. I have been in situation where I needed a quick way to capture and send traffic at the same time. There are ways to configure full packet capture nodes to capture and send the traffic to a master collector, however the goal is a host based collection for cloud servers or temporary capture for incident response or network troubleshooting.

Most of the existing methods approach the problem from an engineering prospective for reliable and permanent full packet capture solution. However, when it comes to a quick way to get PCAP from a remote host, it becomes a tedious task with many manual configuration to be set before receiving the first packet.

Tools that monitor traffic and send aggregated logs are not useful in cases where full packet data is required for deep analysis or file extraction. The tool is intended be to used mainly for capturing traffic from compromised machines or for troubleshooting remote servers as soon as the agent is executed.

Design

Agent to send captured packets
Collector server to receive the packets from the agent
BPF support
Capture stops by condition based on number of packet or size.
Multiple machine capture support
Arkime full packet capture platform support at the collector server

The tools idea is simple, any library that provide full packet capture feature can be used to stream the packets after performing some sort of serialization. Golang programming language has a robust packet processing libraries called GoPacket, in addition it’s a compiled language and that makes it perfect as a start. Python has also strong packet processing packages, but will be a challenge to run a python script on a remote machine with out going through compiling python script into exe file.

Two main libraries are used to achieve the goal, GoPacket as a packet processing library and gRPC for sending the serialized packets to a remote destination.

Implementation

gRPC uses Proto3 (Protocol Buffers) language to describe the services that will be implemented in the server and client. Writing Proto3 services file will enable developers to auto-generate clients for any language supported by gRPC without writing the actual code.

img source

The tool is fairly small, it has only 3 messages and one service consisting of two functions. Details of how to write proto3 can be found in the Proto3 language guide.

syntax = "proto3";
package service;
option go_package = "service/;service";

message Packet{
    bytes Data = 1; 
    bytes Seralizedcapturreinfo = 2; 
}

message EndpointInfo{
    string Hostname = 1;
    string IPaddress = 2;
    string Interface = 3;
}

message Empty {
    string okay = 1;
}

service RemoteCaputre {
    rpc Capture (stream Packet) returns (Empty) {}
    rpc GetReady(EndpointInfo) returns (Empty)  {}

}

Using above proto3 code, a client can be generated containing golang code required to send and receive data between client and server. Google Protobuf compiler generated 470 lines of code that has all code necessary to exchange messages between client and server, the code must be imported by the client and server code being written around the generated Protbuf code. Applications with few functions will have thousands of lines automatically generated with zero effort. The same can be generated for any language.

GoPacket can capture traffic directly from the physical network interface, acting like tcpdump inside the code. GoPacket library has examples folder in the official github repository which shows many use cases for using GoPacket.

Windows NPF interface naming

Linux has simple ifconfig command that shows the interface names such as eth0, eth1 …etc, but Windows has more difficult ways to know what is the name of the interface. In order to capture packets from windows, you need to know the name of the interface which is not what it’s shown in the output of ipconfig or getmac. Golang and Windows Network Interfaces post discuss the naming in more details.

Luckily there is pcap.FindAllDevs() function in GoPacket to find all the physical interfaces. It’s the best option to leave the user decides what interface to capture by mapping the NPF naming with with the friendly names such as wireless or ethernet.

if *listNICsOption {
  count := 0
  devices, err := pcap.FindAllDevs()
  if err != nil {
    log.Fatal(err)
  }
  fmt.Println("Devices found:")
  if OS == "Windows" {
    fmt.Println("")
    for _, device := range devices {
      count++
      fmt.Printf("(%d)- %s:%s\n", count, device.Description, device.Name)
    }
    os.Exit(0)
  } else {
    for _, device := range devices {
      count++
      fmt.Printf("(%d)- %s\n", count, device.Name)
    }
    os.Exit(0)
  }
}

Output of above code will show the interfaces in numbers so the user can select the correct number, same approach is used by tcpdump tool.

Packet Capture

The code to capture is a more complex than below one because of the feature needed such as verbose logging, size and packet tracking, error handling …etc, but for a show case, it’s as simple as below one.

handle, err = pcap.OpenLive(deviceName, snapshotLen, promiscuous, timeout)
if err != nil {
  fmt.Printf("Error opening device %s: %v", deviceName, err)
  os.Exit(1)
  }

  defer handle.Close()
  packetSource := gopacket.NewPacketSource(handle, handle.LinkType())

  if err != nil {
    panic("No interface found")
  }

  packets := packetSource.Packets()

  for {

    select {
    case packet := <-packets:
      if packet.NetworkLayer() == nil || packet.TransportLayer() == nil {
        continue
      }

      data := packet.Data()

      byteArray, err := json.Marshal(packet.Metadata())
      if err != nil {
        fmt.Println(err)
      }
      pkt := service.Packet{
        Data:                  data,
        Seralizedcapturreinfo: byteArray,
      }

      sendchan <- pkt
    }

  }

Json marshal is used to serialize packet data and Metadata into bytes for the gRPC transfer. Golang channels passes data between code snippets. Using channels in the code is enough for Golang endless love.

In the same code, network handling and packet manipulation can be achieved by modifying layers and protocol events which is not the topic of this post.

Sending packets over gRPC

conn, err := grpc.Dial(*serverIP+":9000", grpc.WithInsecure(), grpc.WithBlock())
if err != nil {
  log.Fatalf("can not connect with server %v", err)
}
defer conn.Close()

ctx, cancel := context.WithTimeout(context.Background(), time.Minute)
defer cancel()

// create gRPC client
client := service.NewRemoteCaputreClient(conn)

hostname, _ := os.Hostname()
IP, err := GetIpByInterface(deviceName)
if err != nil {
  fmt.Println(err.Error())
  os.Exit(1)
}

e := service.EndpointInfo{IPaddress: IP, Hostname: hostname, Interface: deviceName}
_, err = client.GetReady(ctx, &e)
if err != nil {
  fmt.Println(err)
}

ServerStream, err := client.Capture(context.Background())
if err != nil {
  log.Fatalf("open stream error %v", err)
}

go func() {
  for {
    select {
    case pkt := <-sendchan:
      err = ServerStream.Send(&pkt)
      if err == io.EOF {
        fmt.Printf("\nReceived EOF: %v\n", err)

      } else if err != nil {
        log.Fatalf("can not send %v", err)
      }
    }
  }
}()

BPF filters

The agent can be converted into a systemd service or windows service for constant transmission, caution needed to not over load the network when capturing multiple servers and sending the traffic to one destination. BPF should be used to narrow down the scope of the capture.

BPF filters can be compiled and used with GoPacket to avoid capturing unnecessary traffic. I rely on Alexa top 1000 website rank + organization internal and business related domains in one huge BPF.

if *whitelisting || *captureFilter != "" {
  buildFilter()
  if err := handle.SetBPFFilter(whitelistFilter); err != nil {
    log.Fatal(err)
  }

} else {
  fmt.Println(*captureFilter)
  whitelistFilter = fmt.Sprintf("not host %s", *serverIP)
  if err := handle.SetBPFFilter(whitelistFilter); err != nil {
    log.Fatal(err)
  }

}

BPF construction

It may be needed to resolve domains to IP Addresses if the supplied whitelist is not in the form of IP Addresses, The list is fetched from server during the initialization of the agent.

func buildFilter() {
  count := 0
  resp, err := http.Get(fmt.Sprintf("http://%s:8080/exceptions.list", *serverIP))
  if err != nil {
    log.Fatalln(err)
  }

  // don't capture yourself 
  whitelistFilter = fmt.Sprintf("not host %s ", *serverIP)

  scanner := bufio.NewScanner(resp.Body)
  defer resp.Body.Close()

  for scanner.Scan() {
    line := scanner.Text()

    if valid.IsDNSName(line) {
      whitelistFilter += fmt.Sprintf("and not host %s ", line)
  }

  whitelistFilter += *captureFilter
}

Collector server implementation

Writing the received packets to disk will not be useful if not supported by a searching platform. The first thought was to use Security Onion and schedule a cron job that will tcpreplay the packets every N minutes. Fortunately I found an interesting option in Arkime (formerly Moloch) to monitor a folders for pcaps and automatically ingest the packets copied to the folder.

gRPC server

gRPC server listens on TCP Port 9000, file server part is used to start a server for public folder where it hosts couple of txt files that acts like settings api. Agents will read the files to know what BPF filters to apply from the public folder.

lis, err := net.Listen("tcp", "0.0.0.0:9000")
if err != nil {
  log.Fatalf("failed to listen: %v", err)
}

// Serve the exception list over http
go func() {

  fs := http.FileServer(http.Dir("./public"))
  http.Handle("/", fs)

  log.Println("Listening on :8080...")
  err := http.ListenAndServe(":8080", nil)
  if err != nil {
    log.Fatal(err)
  }
}()

grpcserver := grpc.NewServer()
service.RegisterRemoteCaputreServer(grpcserver, &Server{})
fmt.Println("Server started. ")
if err := grpcserver.Serve(lis); err != nil {
  log.Fatalf("failed to serve: %v", err)

}

Creating a systemd service to automatically start Moloch capture binary with options to monitor a specific folder for any incoming pcap

The received packets will be written to disk using a special GoPacket writer.

go func() {
  for {
    pkt, err := srv.Recv()
    if err != nil {
      StreamEnd <- true
      break
    }
    metadata := gopacket.PacketMetadata{}
    err = json.Unmarshal(pkt.Seralizedcapturreinfo, &metadata)
    if err != nil {
      fmt.Printf("Error unmarshal the packet %s \n", err)
      continue
    }
    err = w.WritePacket(metadata.CaptureInfo, pkt.Data)
    if err != nil {
      fmt.Println(err)
    }
    endpoints[endpoint].Packetcount++
    fmt.Printf("Received...\nPacketCount: %d ", endpoints[endpoint].Packetcount)
  }
}()

Packets should appear in Arkime platform for packet analysis or download.

Using Structured Analytics Techniques in SOC

2021-10-13T00:00:00+00:00

I keep trying to involve SATs (structured analytic techniques) in my daily Cyber security analysis work. I also keep looking for ways to measure the improvement of SATs on the thinking process used to tackle or analyze cyber incidents.

There really few publications on the topic with direct relation to cyber security. One example I found was a book by Sarah Miller “Cases in Intelligence Analysis: structured Analytic Techniques”, The book practically apply SATs techniques in real world incident as examples happened in the past to illustrate the use of the techniques. The techniques used in the book used are from another book Structured Analytic Techniques by Richards J. Heuer Jr. and Randolph H. Pherson.

Apart from the great time spent with the book, I found chapter 3: discussing cyber attacks on critical infrastructure.There were three techniques selected by author:

Decomposition and visualization using “getting started checklist technique”
Assessment of Cause and Effect using “key assumptions check technique”
And, challenge analysis using “devil’s advocacy technique”

Adding to the above three techniques, I found Analysis of Competing Hypotheses very much applicable to cyber security.

Getting started in SATs for cyber security using these techniques is a good step towards improving SOC personnel thinking, the challenge will remain in measuring the improvement.

Releasing Qradar101 - SIEM Case Investigation

2021-01-30T00:00:00+00:00

I have released a challenge based on Qradar SIEM consisting of 24 questions. The challange is hosted at Cyberdefenders platform. I tried as much as possible to be realistic in the scenario and mimic a real world incident. Almost all the questions can be solved by digging into sysmon logs.

Dataset

Sysmon - swift on security configuration
Powershell logging
Windows Eventlog
Suricata IDS
Zeek logs (conn, HTTP)

The scenario

A financial company was compromised, and they are looking for a security analyst to help them investigate the incident. The company suspects that an insider helped the attacker get into the network, but they have no evidence.

The initial analysis performed by the company’s team showed that many systems were compromised. Also, alerts indicate the use of well known malicious tools in the network. As a SOC analyst, you are assigned to investigate the incident using QRadar SIEM and reconstruct the events carried out by the attacker.

Qradar101 - SIEM Case Investigation

Perception in intelligence analysis

2019-08-30T00:00:00+00:00

By definition, Perception is the interpretations of sensory information in order to represent and understand the presented information, or the environment. It is the process of inference in which people construct their own version of reality on the basis of information provided through the five senses.

Basically, perception is built upon information we already consumed through our senses. It is strongly influenced by education, media, social values, culture ..etc. We tend to perceive what we expect to perceive. Intelligence analysts conclude results to what they expect based on their perception, or they don’t pay attention to details which are not matching the perception they are used to.

Fresh analyst with little perception about the case sometimes help better than experience and old analyst due to the fact that, mind of old analyst resists change. Amount of data and time spent have a proportional relationship to perception. The more data you collect the more perception your brain develops.

Reference:

https://www.cia.gov/library/center-for-the-study-of-intelligence/csi-publications/books-and-monographs/psychology-of-intelligence-analysis/art5.html

Best Quotes I Use in my Infosec Slides

2019-04-20T00:00:00+00:00

“There is noting as useful as a good theory”
—Kurt Lewin

“Knowing yourself is the beginning of all wisdom.”
—Aristotle

“…information consumes the attention of its recipients. Hence a wealth of information creates a poverty of attention.”
—Herbert A. Simon

“When everything is intelligence—nothing is intelligence.”
—Wilhelm Agrell University of Lund, Sweden

“If you know the enemy and know yourself, you need not fear the result of a hundred battles. If you know yourself but not the enemy, for every victory gained you will also suffer a defeat. If you know neither the enemy nor yourself, you will succumb in every battle.”
—Sun Tzu The Art of War

“If you can’t explain it simply, you don’t.” understand it well enough.”
—Albert Einstein

“One may know how to conquer without being able to do it.”
—Sun Tzu

“We kill people based on metadata.”
—General Michael Hayden, former Director of NSA

“To know your Enemy, you must become your Enemy.”
—Sun Tzu

“The world is full of obvious things which nobody by any chance ever observes.”
—Sherlock Holmes

“So in war, the way is to avoid what is strong, and strike at what is weak.”
—Sun Tzu

“We should not only use the brains we have but all that we can borrow.”
― Woodrow Wilson

“Never interrupt your enemy when he is making a mistake.”
― Napoleon Bonaparte.

“Who controls the past, controls the future…”
—George Orwell

“Quickness is the essence of the war.”
—Sun Tzu

“Keep your friends close, and your enemies closer.”

“If you don’t know where you are going, you’ll end up some place else.”
― Yogi Berra

“Anyone can hold the helm when the sea is calm.”
—Publilius Syrus

“The most reliable way to predict the future is to create it.”
—Abraham Lincoln

“By heaven, I’ll make a ghost of him that lets me.”
—Hamlet, William Shakespeare

“Quickest way to find the needle… burn the haystack.”
—Kareem Said

“Computers are useless. They can only give you answers.”
—Pablo Picasso

“The wise warrior avoids the battle.”
—Sun Tzu

“Everybody has a plan until they get punched in the face.”
—Mike Tyson

“…a vision without the ability to execute it is probably an hallucination.”
—Stephen Case

“who wishes to fight must first count the cost.”
—Sun Tzu

Zeek Notes

2019-04-19T00:00:00+00:00

I have been using Zeek for my personal projects for sometime, I will write some Notes for using it and plan to update this list as I get into the advanced capabilities of Zeek.

Note 1

When you write Zeek scripts, test them by executing in the command line. Use the command bro -i interface scriptname.bro You will be able to debug them on the fly, also you can see the errors if any in reporter.log file.

Note 2

When you prepare your intel feeds to be used with Zeek IDS, you will have to make them tab separated (delimiter). It will be very difficult to know where you missed one tab, since nano, gedit and many editors have different tab distance between words you will go crazy trying to find out what is not right in your intel file. To be able to spot the missing tab easily use the command cat -T feed.txt.

Note 3

Set expiry time for intel feeds if you are reading them from threat intelligence source, and set a cron job to periodical read the source looking for new entries.

Note 4

In the feed file, if you want to log all intel hits, https URLs won’t work ( I had hard times on this, maybe only me) You should make the intel type Intel::DOMAIN and provide the indicator without http or https.

Example:

#fields indicator indicator_type meta.source meta.desc meta.url

bad.com Intel::DOMAIN mysecret_source secretIntel.com

Above URL with https and http will log the event in intel.log

Note 5

If you have a bad URL in your feeds, you will notice intel.log will log many entries and that is normal because it logs each request and each response and some other things, see the meta.if_in. You can limit the number of entries in the log by specifying the meta.if_in in your feeds file header and limit it to one thing. Another option is to use the meta.do_notice which tells Zeek IDS if you get a match log this also in notice.log. You will get only one entry in notice.log.

Note 6

you can use Zeek IDS as a parsing tool for offline pcaps. I use following command to parse pcaps and generate logs.

bro -r file.pcap script.bro

Note 7

In real production environment you may have to deploy your IDS properly and use network tap or span port to get the traffic to the IDS. Sensor location in the network is critical, have a look on the white paper Network IDS & IPS Deployment Strategies.

Note 8

if you need to generate logs and you don’t have access to your system you could do that online https://try.bro.org/ don’t do that with pcap that has sensitive data.

Note 9

Now you can use Brim tool to generate and analyze Zeek logs. It supports Zeek and Suricata. The tool basically reads a pcap file and parse it to zeek log and scan it with Suricata.