Problem/Motivation

Fetching OpenStreetMap (OSM) spatial data via the Overpass API is useful for temporary visualization in overpass_api_ui(waiting for the #3610872 to be fixed), but site builders often need to persist this data as native Drupal Entities (e.g., Nodes, Taxonomy Terms, Custom Entities).

Currently, there is no bridge between the Query Builder GUI (overpass_api_ui) and Drupal's import ecosystem. Users can build and preview complex spatial queries visually, but they cannot directly send those query results into an automated import/sync workflow without manually writing custom migration scripts or re-entering raw queries elsewhere.

Steps to reproduce

1. Open the Query Builder GUI provided by overpass_api_ui and visually design a spatial query (e.g., using BBox drawing or Key/Value filters).
2. Execute and preview the results on the map/table.
3. Observe that there is no button or action to pass this visually constructed query directly into Feeds to map its fields to Drupal Entities.

Proposed resolution

Add a new submodule, overpass_api_feeds, that connects overpass_api_ui directly with the Feeds module (drupal/feeds).

The key workflows and components will include:
- UI Action Integration: Add a "Save as Feed Source" (or "Create Feed") action directly inside the overpass_api_ui QueryBuilderForm. Clicking this will automatically transfer the generated query parameters/OQL into a new Feed creation workflow.
- OverpassFetcher Plugin: A Feeds Fetcher plugin that consumes queries originated from the UI or saved query configurations.
- OverpassJsonParser Plugin: A Feeds Parser plugin that flattens the Overpass JSON element tree (nodes, ways, relations), assigns unique composite identifiers (e.g., type/id to prevent collisions), and formats spatial geometries (lat/lon, GeoJSON, WKT) for seamless mapping to modules like Geofield in the standard Feeds mapping UI.

Remaining tasks

- Scaffold overpass_api_feeds.info.yml depending on overpass_api_ui and feeds.
- Implement the "Save as Feed Source" action/submit handler in overpass_api_ui QueryBuilderForm to initiate a Feed entity with pre-filled query data.
- Implement OverpassFetcher Feeds plugin to handle query execution during feed import runs.
- Implement OverpassJsonParser Feeds plugin to map Overpass element attributes (tags, geometry, type, id) to Feeds mapping targets.
- Ensure spatial formats (WKT / GeoJSON) are exposed to the Feeds target pipeline for Geofield compatibility.
- Add functional tests covering the flow from overpass_api_ui query creation to Feeds mapping and entity generation.
- Write user documentation for the end-to-end import workflow.

User interface changes

- A new button/action on the overpass_api_ui QueryBuilderForm: "Save as Feed Source" (or "Import via Feeds").
- New Feeds Fetcher ("Overpass API Fetcher") and Parser ("Overpass API JSON Parser") options in the Feeds Feed Type administration forms.
- Normalized OSM attributes (e.g., osm_id, osm_type, latitude, longitude, wkt_geometry, dynamic tags) exposed in the Feeds Mapping UI.

API changes

- New Feeds Plugins:
- @FeedsFetcher(id = "overpass_api_fetcher")
- @FeedsParser(id = "overpass_api_json_parser")
- Form alter / action integration for overpass_api_ui's QueryBuilderForm to hand off query payload to Feeds.

Data model changes

None. Utilizes existing Feeds configuration and mapping architecture.

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

orkutmuratyilmaz created an issue. See original summary.

orkutmuratyilmaz’s picture

Assigned: Unassigned » orkutmuratyilmaz
Status: Active » Needs work

  • orkutmuratyilmaz committed bf03195b on 1.0.x
    Issue #3614002: Require geofield for development, the feeds kernel test...

  • orkutmuratyilmaz committed a698bd32 on 1.0.x
    Issue #3614002: Run the feeds kernel test in separate processes, as core...

orkutmuratyilmaz’s picture

Status: Needs work » Fixed

Merged MR !13 into 1.0.x. Thanks for the earlier work on the fetcher, @tbcs.

The new overpass_api_feeds submodule (needs Feeds 3.x and overpass_api_preset) provides:

  • an Overpass API fetcher: each feed runs a preset, or Overpass QL of its own with a bounding box for {{bbox}};
  • an Overpass API JSON parser: one item per node, way and relation, keyed by node/123, with tags, the meta fields, and the geometry as WKT and GeoJSON for Geofield;
  • "Save as feed source" in the query builder, which saves the query as a preset and creates a feed that runs it;
  • a list of the affected feeds on the preset delete form.

Large responses are streamed and decoded once, and the files are removed after the import. One thing to watch: Feeds defaults a feed type to an hourly import, so pick a longer period when you use the public Overpass instances. The submodule README covers the setup.

For #3621794: Refactor on top of overpass_api: the fetcher works with any parser, so the openstreetmap Feeds submodule can pair it with its own osm_json parser if it keeps one.

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.