VS Code Extensions¶

While VS Code offers excellent built-in features and a rich marketplace, our ETL workflow has specific needs that generic extensions can't address. These extensions save time and reduce errors by understanding the ETL project structure and conventions.

Custom VS Code extensions enhance your ETL development workflow by automating repetitive tasks, improving navigation, and catching code quality issues early.

Installing extensions¶

First-time setup¶

To install all custom extensions:

make vsce-sync

This installs both marketplace extensions (Ruff, YAML) and our custom extensions from vscode_extensions/*/install/*.vsix.

Updating extensions¶

After compiling an extension, force-reinstall all extensions with their latest versions:

make vsce-sync

This command force-reinstalls all custom extensions, ensuring you get the latest compiled versions even if they're already installed.

Available extensions¶

Find Latest ETL Step¶

Navigate ETL steps efficiently by searching only the latest version of each step.

Use case: You want to open the garden step for energy/electricity_mix but there are multiple versions (2023-01-15, 2023-06-20, 2024-02-10). This extension shows only the latest version files.

Clickable DAG Steps¶

Turn DAG entries into clickable links with visual status feedback:

• 🟢 Up-to-date, no issues
• 🟡 Older version exists, no issues
• ⚪ Archived step
• 🔴 Issues detected

Use case: While editing dag/energy.yml, you want to check the implementation of a dependency. Click the step URI to jump directly to the Python file.

Run Until Cursor¶

Execute code from the start of run() function up to your cursor position in the Interactive Window.

Use case: You're developing a garden step and want to test the first 10 lines of run() without executing the entire function.

Compare Previous Version¶

Quickly compare the current file with its previous version.

Use case: You're reviewing changes in energy/2024-02-10/electricity_mix.py and want to see what changed from 2023-06-20.

DoD Syntax¶

Work seamlessly with DoD references in YAML and Python files.

Use case: You're documenting indicator metadata and need to reference DoD entries. Start typing #dod: and get autocomplete suggestions with descriptions.

Detect Outdated Practices¶

Catch outdated code patterns as you type with configurable warnings.

Use case: You copy code from an old step that uses dest_dir. The extension highlights it with a warning: "Use paths.create_dataset instead."

Detected patterns:

• dest_dir usage → Use modern path handling
• geo.harmonize_countries() → Use paths.regions.harmonize_names(tb)
• paths.load_dependency() → Use paths.load_dataset() or paths.load_snapshot()
• if __name__ == "__main__" in snapshots → Remove this outdated pattern

Chart Preview¶

Preview ETL datasets and charts directly in VS Code without leaving your editor.

Use case: You're working on a garden step and want to quickly check the indicators it produces — their sparklines, value ranges, null rates, and data quality flags — without running the full pipeline or opening a browser.

Works for:

• Garden / meadow .py steps → opens a dataset preview panel with indicator cards
• .chart.yml files and multidim steps → embeds the staging chart viewer

Dataset preview¶

Opens a live panel showing:

• Indicator cards sorted by popularity (most-viewed indicators first)
• Sparkline charts per entity, entity selector, and shuffle button
• Stats: year range, entity count, min/max values, null rate
• Data quality flags: missing title, unit, description, or origins
• Popularity score (★) sourced from the analytics_popularity table

The panel auto-rebuilds when you save the step file (uses etlr --watch in the background).

Chart preview¶

Embeds the staging admin chart viewer for .chart.yml and multidim export steps.

Developing extensions¶

Improving existing extensions or creating a new ones is simple. Below, we outline the basic workflow for extension development.

Edit the extension¶

Edit code in vscode_extensions/[extension-name]/src/extension.ts

Compile the extension¶

Run the following command from the ETL root directory:

make vsce-compile EXT=extension-name

This compiles the extension and packages it into a .vsix file located in vscode_extensions/[extension-name]/install/.

Optionally, you can add BUMP=patch (or minor/major) to increment the version in package.json. And INSTALL=1 to install the compiled extension into your local VS Code.

Sync extensions¶

Install latest versions (including the one you just compiled) of all extensions:

make vsce-sync

Reload VS Code¶

Reload VS Code: cmd+shift+p → "Developer: Reload Window"

Troubleshooting¶

Extension not working after update¶

Problem: Ran make vsce-sync but extension didn't update.

Cause: This command skips already-installed extensions.

Solution: Use make vsce-sync to force-reinstall all extensions:

make vsce-sync

Then reload VS Code: Cmd+Shift+P → "Developer: Reload Window"

Changes not taking effect¶

Problem: Modified extension code but changes don't appear in VS Code.

Solution: Compile and reinstall the extension:

# Quick method
make vsce-compile EXT=extension-name INSTALL=1

# Or compile then sync
make vsce-compile EXT=extension-name
make vsce-sync

Then reload VS Code: Cmd+Shift+P → "Developer: Reload Window"

Manual troubleshooting¶

If the automated commands don't work, try manual reinstallation:

cd /path/to/etl
code --install-extension vscode_extensions/[extension]/install/[extension]-[version].vsix --force

Then reload VS Code: Cmd+Shift+P → "Developer: Reload Window"

Extension conflicts¶

Solution: Check for conflicting keybindings in VS Code settings:

File → Preferences → Keyboard Shortcuts → Search for the shortcut

Updating dependencies¶

See the extension development README for guidance on updating npm dependencies, handling security vulnerabilities, and understanding when to bump extension versions.