<include>

Overview

The processor imports and executes external WebHarvest configuration files, enabling modular scraper design, code reuse, and separation of concerns. Included configurations execute in the same context, sharing variables and state.

Usage Examples

Example 1: Include common functions

example-1.xml

<!-- main.xml -->
<config>
  <!-- Import shared functions -->
  <include path="functions/http-helpers.xml"/>
  <include path="functions/data-extractors.xml"/>
  
  <!-- Use imported functions -->
  <call name="fetchAndParse">
    <call-param name="url">https://example.com</call-param>
  </call>
</config>

Example 2: Modular configuration

example-2.xml

<!-- main.xml -->
<config>
  <include path="config/constants.xml"/>     <!-- API keys, URLs -->
  <include path="scrapers/products.xml"/>    <!-- Product scraping logic -->
  <include path="scrapers/reviews.xml"/>     <!-- Review scraping logic -->
  <include path="export/formatters.xml"/>    <!-- Export functions -->
</config>

Example 3: Conditional include

example-3.xml

<?xml version="1.0" encoding="UTF-8"?>
<config xmlns="http://org.webharvest/schema/2.1/core">
<def var="environment" value="production"/>

<if condition="${environment == 'production'}">
  <include path="config/prod-settings.xml"/>
</if>
<else>
  <include path="config/dev-settings.xml"/>
</else>
</config>

Important Notes

Path resolution:: Paths are relative to working directory or absolute
Shared context:: Included config runs in same context, shares variables
Execution timing:: Included config executes immediately at include point
Recursive includes:: Can include files that include other files