Buffers

Represent and manipulate raw binary data in Node.js with the Buffer class

Overview

JavaScript was originally designed for the browser where it primarily dealt with text: HTML, CSS, JSON. But Node.js runs on servers where you often need to handle raw binary data — reading image files, receiving TCP packets, computing checksums, or working with binary protocols. The `Buffer` class was created to fill this gap before TypedArrays became widely available in JavaScript. A Buffer is a fixed-size chunk of memory allocated outside the V8 heap. It holds raw bytes, making it perfect for I/O operations. When you read a file without specifying an encoding, you get a Buffer. When data arrives on a network socket, you get chunks as Buffers. When you call a crypto function, it returns a Buffer. Understanding Buffers is essential for working with Node's lower-level APIs. Buffers are closely related to JavaScript's `Uint8Array` — in fact, `Buffer` is a subclass of `Uint8Array`. You can slice them, concatenate them, convert to and from strings with various encodings (utf8, hex, base64, latin1), and compare them. The key constraint is that Buffers have a fixed size once allocated — if you need dynamic growth, you concatenate multiple Buffers. Modern Node.js code often uses Buffers implicitly — the `fs`, `crypto`, `net`, and `http` modules all use them internally. You interact with Buffers most directly when: parsing binary file formats, implementing network protocols, encoding data for transmission (base64 for HTTP, hex for hashing), or optimising memory usage by avoiding string creation for binary data.

Key Points

•Buffer is a fixed-size array of bytes, allocated outside the V8 heap
•Buffer.from(string, encoding) converts a string to bytes; buf.toString(encoding) reverses it
•Buffer.alloc(size) creates a zero-filled buffer; Buffer.allocUnsafe(size) is faster but uninitialized
•Buffer.concat([buf1, buf2]) combines multiple buffers into one
•Common encodings: "utf8" (default), "hex", "base64", "latin1" ("binary")
•Buffer is a subclass of Uint8Array — you can use array indexing and typed array methods on it

Examples

Creating Buffers from strings and converting back

// From a UTF-8 string
const buf1 = Buffer.from('Hello, Node.js!', 'utf8');
console.log(buf1);           // <Buffer 48 65 6c 6c 6f 2c ...>
console.log(buf1.length);    // 15 (bytes, not characters)
console.log(buf1.toString()); // Hello, Node.js!

// Encoding conversions
console.log(buf1.toString('hex'));    // 48656c6c6f2c204e6f64652e6a7321
console.log(buf1.toString('base64')); // SGVsbG8sIE5vZGUuanMh

// Decode hex back to string
const buf2 = Buffer.from('48656c6c6f', 'hex');
console.log(buf2.toString('utf8')); // Hello

Allocating and writing to Buffers

// alloc: zero-filled, safe
const safe = Buffer.alloc(4);
console.log(safe); // <Buffer 00 00 00 00>

// Write individual bytes by index
safe[0] = 0xFF;
safe[1] = 0x00;
safe[2] = 0x7E;
safe[3] = 0x01;
console.log(safe); // <Buffer ff 00 7e 01>

// allocUnsafe: faster but may contain garbage data — always fill before reading
const unsafe = Buffer.allocUnsafe(10);
unsafe.fill(0); // manually zero it out
console.log(unsafe);

Never expose data from Buffer.allocUnsafe to users without filling it first — it can contain previously freed memory.

Concatenating Buffers from a stream of chunks

// Simulating collecting chunks from a stream
const chunks = [
  Buffer.from('Hello'),
  Buffer.from(', '),
  Buffer.from('World'),
  Buffer.from('!'),
];

const combined = Buffer.concat(chunks);
console.log(combined.toString('utf8')); // Hello, World!
console.log(combined.length);           // 13

Never concatenate buffers with the + operator — it converts to strings. Always use Buffer.concat().

Using Buffers for base64 encoding/decoding (common for HTTP and data URIs)

// Encode binary data as base64 for HTTP transmission
function toBase64(data) {
  return Buffer.from(data, 'utf8').toString('base64');
}

function fromBase64(encoded) {
  return Buffer.from(encoded, 'base64').toString('utf8');
}

const original  = 'secret data: £100';
const encoded   = toBase64(original);
const decoded   = fromBase64(encoded);

console.log('Encoded:', encoded);  // c2VjcmV0IGRhdGE6IMKjMTAw
console.log('Decoded:', decoded);  // secret data: £100
console.log('Match:', original === decoded); // true

Common Mistakes

Using Buffer.allocUnsafe() without filling with .fill(0) — the memory may contain sensitive data from previous allocations
Concatenating Buffers with the + operator (buf1 + buf2 converts them to strings first, corrupting binary data)
Assuming buf.length equals character count — for multi-byte Unicode, byte count > character count
Using the deprecated `new Buffer()` constructor — it was removed for security; use Buffer.from(), Buffer.alloc(), or Buffer.allocUnsafe()

Note

When accumulating chunks from a stream, push each chunk into an array and call Buffer.concat() once at the end — this is far more efficient than repeated string concatenation or repeated Buffer.concat() calls in a loop.

Buffers

Overview

Key Points

Examples

Common Mistakes

Note

Exercises

Knowledge Check

What is the relationship between Buffer and Uint8Array in Node.js?