Skip to main content

Lab 11.1: Process Memory Analysis

Learning Objectives​

By the end of this lab, you will be able to:

  • Use Sysinternals tools to analyze memory
  • Understand the Working Set and different types of memory
  • Identify memory anomalies of a process
  • Detect suspicious processes by their memory behavior

Estimated Duration​

45 minutes

Prerequisites​

  • Windows 10/11 VM
  • Sysinternals tools (Process Explorer, VMMap, RAMMap)
  • Administrator rights

Professional Context​

Memory analysis is used for:

  • Diagnosing memory leaks
  • Identifying fileless malware
  • Analyzing application performance
  • Forensics and incident investigation

Part 1: Overview with Task Manager​

Step 1.1: Open Task Manager​

Ctrl+Shift+Esc

Go to the Performance tab > Memory

Step 1.2: Understand the metrics​

MetricDescription
In useRAM currently used
AvailableFree RAM + standby
Committed"Promised" memory (can > RAM)
CachedData cached for quick access
PagedIn the page file on disk
Non-pagedAlways in RAM (drivers)

Step 1.3: View by process​

Details tab > Right-click on columns > Select columns

Add:

  • Working set (memory)
  • Memory (private working set)
  • Memory (commit size)

Question 1: Which process uses the most memory on your system?


Part 2: In-depth Process Explorer​

Step 2.1: Download and launch​

# Download Sysinternals Suite if not already done
# https://docs.microsoft.com/en-us/sysinternals/downloads/

# Launch Process Explorer as admin
.\procexp64.exe

Step 2.2: Configure memory columns​

  1. View > Select Columns...
  2. Process Memory tab, add:
    • Working Set
    • Private Bytes
    • Virtual Size
    • Peak Working Set
    • Page Faults

Step 2.3: Analyze a process​

  1. Select a process (e.g., explorer.exe)
  2. Right-click > Properties
  3. Performance tab

Observe:

  • Working Set: Memory currently in RAM
  • Private Bytes: Memory exclusive to the process
  • Virtual Size: Total address space

Question 2: What is the difference between Working Set and Private Bytes?

Step 2.4: View loaded DLLs​

  1. View > Show Lower Pane
  2. View > Lower Pane View > DLLs
  3. Select a process

Question 3: How many DLLs are loaded in explorer.exe?


Part 3: VMMap for Detailed Analysis​

Step 3.1: Launch VMMap​

vmmap64.exe

Select a process to analyze (e.g., notepad.exe - launch it first).

Step 3.2: Understand the view​

VMMap displays the complete address space:

TypeColorDescription
ImageYellowExecutable/DLL files
Mapped FileOrangeMemory-mapped files
ShareableGreenShareable memory
HeapRedProcess heap
StackPurpleThread stacks
Private DataBluePrivate data
FreeWhiteUnallocated

Step 3.3: Identify regions​

  1. Click on a region in the graph
  2. Look at the details below
  3. Note the address, size, protection

Memory protection:

  • R: Read
  • W: Write
  • X: Execute
  • G: Guard page

Step 3.4: Search for anomalies​

Alert signals:

  • RWX: Readable, writable AND executable region (suspicious!)
  • Very large heap: Possible memory leak
  • Unexplained Private Data: Possible injection
# Script to detect suspicious RWX regions
# (Requires specific tools like WinDbg)

Question 4: Are there any RWX regions in notepad.exe? If so, which ones?


Part 4: RAMMap for Global View​

Step 4.1: Launch RAMMap​

rammap64.exe

Launch as administrator.

Step 4.2: Overview​

Use Counts tab:

TypeDescription
Process PrivateMemory exclusive to processes
Mapped FileMapped files
ShareableShareable memory
Page TablePaging tables
Nonpaged PoolNon-pageable kernel pool
Driver LockedLocked by drivers
StandbyCached, freeable if needed
ModifiedModified, not yet written to disk

Step 4.3: File Summary tab​

Shows which files are mapped in memory.

Question 5: Which files consume the most RAM in cache?

Step 4.4: Clear the cache​

Empty menu > Empty Standby List

Caution: Use with care in production!


Part 5: Anomaly Detection​

Step 5.1: Detection script​

# Basic memory analysis script
Write-Host "=== SYSTEM MEMORY ANALYSIS ===" -ForegroundColor Cyan

# Top 10 processes by Working Set
Write-Host "`n[Top 10 Working Set]" -ForegroundColor Yellow
Get-Process | Sort-Object WorkingSet64 -Descending | Select-Object -First 10 |
Format-Table Name, Id, @{N='WorkingSet(MB)';E={[math]::Round($_.WorkingSet64/1MB,2)}}

# Top 10 by Private Bytes
Write-Host "[Top 10 Private Bytes]" -ForegroundColor Yellow
Get-Process | Sort-Object PrivateMemorySize64 -Descending | Select-Object -First 10 |
Format-Table Name, Id, @{N='PrivateBytes(MB)';E={[math]::Round($_.PrivateMemorySize64/1MB,2)}}

# Processes with suspicious ratio (Virtual >> Working Set)
Write-Host "[Suspicious Virtual/Working Ratio]" -ForegroundColor Yellow
Get-Process | Where-Object {
$_.WorkingSet64 -gt 10MB -and
$_.VirtualMemorySize64 -gt ($_.WorkingSet64 * 100)
} | Format-Table Name, Id,
@{N='Working(MB)';E={[math]::Round($_.WorkingSet64/1MB,2)}},
@{N='Virtual(MB)';E={[math]::Round($_.VirtualMemorySize64/1MB,2)}}

# Global system info
Write-Host "[System Info]" -ForegroundColor Yellow
$os = Get-CimInstance Win32_OperatingSystem
Write-Host " Total RAM: $([math]::Round($os.TotalVisibleMemorySize/1MB,2)) GB"
Write-Host " Free RAM: $([math]::Round($os.FreePhysicalMemory/1MB,2)) GB"
Write-Host " Total Commit: $([math]::Round($os.TotalVirtualMemorySize/1MB,2)) GB"

Step 5.2: Malware indicators​

Suspicious signs in memory:

  • Process with standard name but unusual location
  • RWX regions (executable heap/stack)
  • Injection into legitimate processes
  • "Hollow" processes
# Check process paths
Get-Process | Where-Object { $_.Path } | ForEach-Object {
$expectedPaths = @("C:\Windows", "C:\Program Files", "C:\Program Files (x86)")
$isExpected = $expectedPaths | ForEach-Object { $proc.Path -like "$_*" } | Where-Object { $_ }

if (-not $isExpected -and $_.Name -in @("svchost", "csrss", "services", "lsass")) {
Write-Host "[SUSPECT] $($_.Name) - Path: $($_.Path)" -ForegroundColor Red
}
}

Question 6: Did you find any suspicious processes?


Part 6: Practical Exercise​

Step 6.1: Create a memory leak (simulation)​

# Script that simulates a memory leak
# TO BE RUN IN A TEST VM

$leak = @()
Write-Host "Simulating memory leak..." -ForegroundColor Yellow
Write-Host "Press Ctrl+C to stop"

while ($true) {
$leak += [byte[]]::new(10MB) # Allocate 10MB
$totalMB = $leak.Count * 10
Write-Host "Allocated memory: $totalMB MB" -ForegroundColor Gray
Start-Sleep -Seconds 1

if ($totalMB -ge 500) {
Write-Host "Limit reached, stopping." -ForegroundColor Red
break
}
}

# Release
$leak = $null
[System.GC]::Collect()
Write-Host "Memory released." -ForegroundColor Green

Step 6.2: Observe with Process Explorer​

  1. Run the leak script
  2. Observe the powershell.exe process in Process Explorer
  3. Note the evolution of the Working Set and Private Bytes

Deliverables​

  1. Screenshots:

    • Task Manager Memory tab
    • Process Explorer with memory columns
    • VMMap of a process
    • RAMMap global view
  2. Answers to the 6 questions

  3. Analysis report (1-2 pages):

    • Most resource-intensive processes
    • Detected anomalies
    • Recommendations

Evaluation Criteria​

CriteriaPoints
Task Manager usage15%
Process Explorer25%
VMMap25%
RAMMap15%
Detection script20%

Reference Commands​

# Process memory
Get-Process | Sort-Object WorkingSet64 -Descending | Select-Object -First 10

# System info
Get-CimInstance Win32_OperatingSystem | Select-Object *Memory*

# Page file
Get-WmiObject Win32_PageFileUsage

Troubleshooting​

Process Explorer does not launch​

  • Run as administrator
  • Check if it's blocked by antivirus

VMMap shows little info​

  • Select the process in the list
  • Ensure the process still exists

Lab developed as part of the Operating Systems course - MODULE 11