# File Descriptors (FD): What They Are, How They Work, and Why You Hit EMFILE

If you’ve ever seen an error like:

```js
EMFILE: too many open files
```

you’ve already met one of the most fundamental pieces of an operating system: the **file descriptor (FD)**.

This blog explains:

*   what an FD is
    
*   how its lifecycle works
    
*   why your system ran out of them during image export
    
*   **how memory differs from FD (second problem you hit)**
    
*   **and how Excel image export actually behaves internally (third problem)**
    

* * *

# 🧠 What is a File Descriptor (FD)?

A **file descriptor** is a small integer that the operating system gives your process when it opens a file or resource.

Think of it as a **ticket number**:

*   You ask the OS: “Open this file”
    
*   OS says: “Here’s FD = 7”
    
*   You use **7** to read/write
    
*   When done → you close it
    

* * *

## 🧾 Example

```js
const fs = require('fs');

const fd = fs.openSync('data.txt', 'r'); // fd = 7 (example)
const buffer = Buffer.alloc(100);

fs.readSync(fd, buffer, 0, 100, 0);
fs.closeSync(fd);
```

* * *

# 📦 What counts as a file?

In Unix systems:

> **Everything is treated like a file**

| Resource | Uses FD? |
| --- | --- |
| Disk file | ✅ |
| HTTP request | ✅ |
| Socket | ✅ |
| Stream | ✅ |

* * *

# 🔄 FD Lifecycle (Step-by-Step)

```text
[1] Request resource
        ↓
[2] OS opens it
        ↓
[3] OS assigns FD (integer)
        ↓
[4] Your app uses FD
        ↓
[5] You close FD
        ↓
[6] OS frees FD
```

* * *

## 📊 Diagram

```text
Application                  Operating System
-----------                 -----------------
   open()   ─────────────▶   Allocate FD (e.g., 5)
                            Open file/socket

   read(fd=5) ◀──────────▶   Read data

   write(fd=5) ◀─────────▶   Write data

   close(5) ─────────────▶   Release FD
                            Free slot
```

* * *

# 🔢 Why FD Limits Exist

Each process has a limit:

```bash
ulimit -n
```

Typical values:

*   1024
    
*   4096
    
*   65535 (tuned systems)
    

* * *

# 💥 What is EMFILE?

```text
EMFILE = Too Many Open Files
```

It means:

> ❗ You opened more resources than allowed at the same time

* * *

# 🧨 Real Example (Your Case)

*   50 concurrent downloads
    
*   Each opens:
    
    *   1 HTTP socket (FD)
        
    *   1 file write (FD)
        

👉

```text
50 × (socket + file) = ~100 FDs
```

With multiple jobs:

```text
5 jobs → ~500 FDs → EMFILE
```

* * *

# ❌ Bad Pattern

```js
await Promise.all(urls.map(download));
```

👉 Opens everything at once

* * *

# ✅ Good Pattern

```js
for (const url of urls) {
  await download(url);
}
```

👉 Controlled FD usage

* * *

# 🧠 NEW TOPIC 1: FD vs Memory (Critical Difference)

This is where many people get confused.

* * *

## 📊 Comparison

| Concept | FD | Memory |
| --- | --- | --- |
| What it is | Handle | Data |
| Managed by | OS | Your app |
| Limit | OS limit (ulimit) | RAM |
| Error | EMFILE | OOM (Out of Memory) |

* * *

## 🔍 Example

```js
const buffer = await fs.readFile('image.jpg');
```

### What happens:

```text
open file → FD used
read file → buffer created in RAM
close file → FD released
```

👉 After this:

*   FD = ❌ gone
    
*   Buffer = ✅ still in memory
    

* * *

## 💥 Key Insight

> ❗ Closing a file does NOT free memory

* * *

## 🧨 Your second problem (memory)

Even after fixing EMFILE:

```js
workbook.addImage({ buffer })
```

👉 ExcelJS stores ALL buffers internally

So:

```text
6000 images × 150KB ≈ 900MB RAM
```

👉 This leads to **memory pressure**, not FD issue

* * *

## 🧠 Simple analogy

*   FD = number of doors open 🚪
    
*   Memory = stuff inside the room 📦
    

Closing doors doesn’t remove the stuff.

* * *

# 🧠 NEW TOPIC 2: Excel Image Rendering Limitation (Your 3rd Issue)

You noticed:

> After ~3000 rows → images overlap at top

* * *

## 🔍 Root cause

Excel calculates image positions using:

```text
Row height → converted to EMU
1pt = 12,700 EMU
```

Each image Y position = **sum of all previous row heights**

* * *

## ⚠️ Limitation

Excel internally uses:

```text
32-bit signed integer
Max = 2,147,483,647
```

* * *

## 📊 Calculation

If:

```text
row height = 50pt
→ 50 × 12700 = 635,000 EMU per row
```

Then:

```text
2,147,483,647 / 635,000 ≈ 3,382 rows
```

* * *

## 💥 What happens after that?

```text
Value overflows → becomes negative
→ Excel places image at wrong position (top)
→ all images overlap
```

* * *

## 📊 Visual

```text
Correct:
Row 1 → Y = 0
Row 2 → Y = 635k
Row 3000 → Y = valid

After overflow:
Row 3500 → Y = negative ❗
→ jumps to top
→ overlap
```

* * *

## ❌ Why your “row height fix” felt bad

You reduced:

```text
row height = 16pt
```

👉 avoids overflow BUT:

*   image still 50px
    
*   row smaller than image
    
*   UI breaks
    

* * *

# 🏁 Proper Solutions (Summary)

## ✅ For FD issue

*   limit concurrency
    
*   avoid massive `Promise.all`
    

* * *

## ✅ For Memory issue

*   compress images
    
*   reduce unique images
    
*   limit concurrent exports
    

* * *

## ✅ For Excel overlap issue

### Best:

👉 Split sheets

```text
Sheet 1 → 0–3000 rows
Sheet 2 → 3001–6000
```

### Alternative:

*   reduce image size
    
*   or fallback to URLs
    

* * *

# 🧠 Final Mental Model

```text
Problem 1 → FD (too many things open at once)
Problem 2 → Memory (too much data kept)
Problem 3 → Excel limit (internal integer overflow)
```

* * *

# 💬 One-line takeaway

> FD issues are about “how many things are open at once”, memory issues are about “how much data you keep”, and Excel issues are about “format limitations you cannot bypass.”