Skip to content

Population Analysis of High-Risk API Calls

High-risk memory-manipulation and process-injection APIs are useful hunting signals, but their presence alone does not establish malicious activity. Debuggers, EDR products, browsers, development tools, and other legitimate software may call the same APIs.

Core principle
API observed ≠ malicious

BASELINE → COMPARE → OUTLIER

Use population analysis to find hosts or processes performing this behavior at a statistically unusual rate compared with similar systems.1

Population-analysis model
Similar host population
  → measure normal API-call frequency
  → establish baseline
  → compare individual hosts and processes
  → identify statistical outliers
  → investigate high-rate injection behavior

Behavior of interest

Analyze individual APIs, related API groups, and sequences. A representative sequence is:

VirtualAlloc / VirtualAllocEx
  → WriteProcessMemory
  → VirtualProtectEx / NtProtectVirtualMemory
  → CreateRemoteThread / NtCreateThreadEx

Do not make detection dependent on this exact sequence. Variants may omit, replace, reorder, or separate operations across a short time window.

WriteProcessMemory allows one process to write into another process's virtual address space. VirtualProtectEx and NtProtectVirtualMemory can change memory permissions, including RW → RX, making newly written code executable. See Process and Memory Hunting: APIs to recognize for the API reference rather than duplicating the glossary here.

Compare like with like

Population analysis is only useful when the peer population is meaningful. Do not create one baseline across fundamentally different systems.

Prefer peer groups such as:

  • Windows workstations
  • Domain controllers
  • Application servers
  • Developer workstations
  • VDI systems
  • Security-tooling hosts
  • Kiosks
  • Systems running the same server application and version

A developer workstation or EDR sensor may legitimately produce behavior that would be extremely unusual on a standard office workstation.

Example
Peer population: 2,000 Windows workstations
Typical:         0–2 WriteProcessMemory-related events/hour
Host A:          1 event/hour
Host B:          0 events/hour
Host C:          175 events/hour

Host C → STATISTICAL OUTLIER → INVESTIGATE

The useful question is not merely "Was the API called?" It is:

Why is this host or process performing this behavior far more often than its peers?

Data source and retained fields

This technique requires endpoint telemetry capable of recording granular API activity or equivalent EDR evidence. Normal Windows Security Event Logs alone generally do not provide this depth.

Potential sources include:

  • EDR endpoint telemetry
  • ETW-derived telemetry
  • Kernel- or user-mode instrumentation
  • API monitoring
  • Specialized endpoint sensors

Retain the following fields where available:

Field Analytical value
timestamp Windows, rates, sequence timing, and time-series comparison
host, host_role, peer_group Entity and correct comparison population
calling_process, calling_process_path Source identity and prevalence
calling_process_hash, calling_process_signer Reputation, first-seen, and trust context
target_process, target_process_path Injection target and target rarity
api_function, api_category Individual, grouped, and sequence analysis
source_pid, target_pid Same-host correlation
process_guid or stable process ID Durable event joins across PID reuse
user User-level rates and context
api_call_count, time_window Rate calculation and aggregation

Analytical dimensions

Calculate counts and rates by combinations such as:

  • API calls per host
  • API calls per process
  • API calls per process and target process
  • API calls per user
  • API calls per host role or peer group
  • API sequences per process
  • Unique target processes accessed

Compare each value with the appropriate peer population and, when useful, with the entity's own historical behavior.

Implementation options

Sophisticated machine learning is not mandatory. The method can use:

  • Statistical baselines
  • Percentiles
  • Standard deviations or z-scores
  • Median and deviation methods, such as median absolute deviation
  • Rare-event analysis
  • Time-series anomaly detection
  • SIEM anomaly-detection or ML capabilities

Select a method appropriate for sparse, often zero-heavy data. Record the baseline window, comparison window, minimum event volume, peer-group definition, and threshold so results are explainable and reproducible.

Model design details

The source method separates the entity being measured from the population supplying context. For example, measure API-call rate by calling process or host, compare it with a peer population, and retain the process, target, and host as explanatory fields.

Design choice Practical use
Feature Count or rate of an API, API category, or ordered sequence
Population Comparable hosts or processes rather than the entire fleet
Bucket span Short enough to expose an injection burst; long enough to avoid unstable sparse counts
Influencer Calling process, target process, host, user, or API that explains the score
Cardinality Use rarity for distinctive combinations; use rate deviation for common names with unusual frequency
Sensitivity Tune the anomaly threshold or contamination setting separately for exploratory hunts and production alerts

A score expresses statistical exceptionality, not maliciousness. Require retained evidence that explains the baseline, the deviation, and the entities responsible.2

Influencers and explanatory fields

When an anomaly occurs, identify which entities contributed most strongly. Useful influencers include:

  • Host
  • Calling process
  • Target process
  • API or API category
  • User
  • Application
  • Host role
Influencer example
Anomaly:   Host X has abnormally high memory-manipulation activity
Influencer: unknown.exe accounts for 93% of the unusual API calls

This is more actionable than knowing only that Host X is anomalous.

Sequence analysis

Individual API frequency is useful, but sequence analysis can provide a stronger signal.

VirtualAllocEx
  → WriteProcessMemory
  → VirtualProtectEx
  → CreateRemoteThread

A host or process executing a suspicious sequence far more frequently than comparable systems should receive substantially more investigative priority. Where feasible, track:

  • Sequence count and frequency
  • Calling process
  • Target process
  • Time between operations
  • Protection transitions
  • Whether the sequence completed with remote-thread or equivalent execution

Hunting and alerting logic

Generate a hunting lead when the frequency of a high-risk API, related API group, or suspicious sequence for a host or process significantly exceeds the calculated or learned baseline for its peer population.

current_rate
  vs
peer_baseline

statistically significant deviation → flag anomaly

Avoid simplistic logic such as:

WriteProcessMemory > 0 → malicious

Prefer a weighted combination:

Abnormal API-call rate
  + unusual calling process
  + unusual target process
  + suspicious API sequence
  + corroborating memory or thread evidence
  = high-value hunting lead

Correlation and investigation

Correlate outliers with:

  • Process lineage
  • Sysmon Event 10 ProcessAccess
  • Sysmon Event 8 CreateRemoteThread
  • Executable MEM_PRIVATE memory
  • RW → RX protection changes
  • Unbacked thread start addresses
  • Image and DLL loads
  • File creation
  • User activity
  • Network connections and DNS
  • Authentication and persistence activity
Investigation flow
Population anomaly
  → host has excessive memory API activity
  → identify calling process
  → identify target process
  → inspect process lineage
  → check injection and memory telemetry
  → check thread start and backing module
  → check network behavior
  → determine whether activity is benign or malicious

First-seen and population analysis

Combine population analysis with first-seen detection:

Previously unseen process
  + rare API sequence
  + high frequency
  + observed on one host
  + MEM_PRIVATE execution
  = substantially stronger lead

Value for unknown threats

This technique does not require a known malware hash, IOC, exploit, or command line. It detects deviation from expected behavior. An unknown in-memory stager or exploit may become visible because it causes a host or process to perform memory-manipulation behavior at an unusual frequency compared with peers.

It is especially useful for:

  • Unknown malware
  • In-memory implants
  • Process injection
  • Exploit activity
  • Zero-day hunting
  • Behavioral detection

Historical implants such as DoublePulsar illustrate the broader concept: in-memory activity may drive elevated memory-writing or memory-protection behavior on systems that normally show little or none. The hunting value comes from the population deviation and supporting evidence, not from attribution alone.

Caveats

False positives can occur when:

  • Developer systems are compared with office workstations
  • EDR or security tools perform legitimate injection-like behavior
  • Software updates change normal behavior
  • Application versions behave differently
  • A new legitimate application is deployed
  • Workload changes alter API frequency

Recalculate baselines when populations or workloads change. Retain enough context to explain why an entity was considered anomalous and to suppress well-understood recurring behavior narrowly rather than weakening the entire technique.

Quick reference

Population analysis
DEFINE PEERS
  → BASELINE BEHAVIOR
  → MEASURE ENTITY
  → COMPARE
  → IDENTIFY OUTLIER
  → CORRELATE
  → INVESTIGATE

Question: Is this host or process performing the behavior far more often than comparable systems, and what evidence explains the deviation?

  1. Gianluca Tiepolo and Dan Sorensen, Advanced Cyber Threat Intelligence and Hunting, first edition, Packt, 2026, Chapter 6, "Memory injection and code manipulation," supplied EPUB extraction lines 361–391. ↩

  2. Tiepolo and Sorensen, Chapter 6, anomaly-model concepts and practical limits, supplied EPUB extraction lines 130–212. ↩