Skip to content

cermak-petr/act-run-crawler-url-list-batch

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

11 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

act-run-crawler-url-list-batch

This act starts crawler runs for a specified list of URLs.

It starts the runs in batches of "N_OF_URLS_IN_BATCH", always waits for one batch to finish before starting the next. After all runs finish, the act returns a list of all ExecutionIDs.

Example input:

{
    "urls": ["URL_1", "URL_2", "URL_3", ...],
    "label": "START_URL_LABEL",
    "batch": "N_OF_URLS_IN_BATCH"
}

Example output:

{
    "executionIds": [
        "EXECUTION_ID_1",
        "EXECUTION_ID_2",
        "EXECUTION_ID_3",
        ...
    ]
}

It is useful to chain output of this act into "petr_cermak/executions_merge" act.

About

Apify act for starting crawler runs for a specified list of URLs.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published