mirror of https://github.com/hwchase17/langchain
Harrison Chase
11 months ago
committed by
GitHub
parent
95bcf68802
commit
aa0e69bc98
@ -1,168 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Apify\n",
|
||||
"\n",
|
||||
"This notebook shows how to use the [Apify integration](/docs/ecosystem/integrations/apify.html) for LangChain.\n",
|
||||
"\n",
|
||||
"[Apify](https://apify.com) is a cloud platform for web scraping and data extraction,\n",
|
||||
"which provides an [ecosystem](https://apify.com/store) of more than a thousand\n",
|
||||
"ready-made apps called *Actors* for various web scraping, crawling, and data extraction use cases.\n",
|
||||
"For example, you can use it to extract Google Search results, Instagram and Facebook profiles, products from Amazon or Shopify, Google Maps reviews, etc. etc.\n",
|
||||
"\n",
|
||||
"In this example, we'll use the [Website Content Crawler](https://apify.com/apify/website-content-crawler) Actor,\n",
|
||||
"which can deeply crawl websites such as documentation, knowledge bases, help centers, or blogs,\n",
|
||||
"and extract text content from the web pages. Then we feed the documents into a vector index and answer questions from it.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"#!pip install apify-client openai langchain chromadb tiktoken"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"First, import `ApifyWrapper` into your source code:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from langchain.document_loaders.base import Document\n",
|
||||
"from langchain.indexes import VectorstoreIndexCreator\n",
|
||||
"from langchain.utilities import ApifyWrapper"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Initialize it using your [Apify API token](https://console.apify.com/account/integrations) and for the purpose of this example, also with your OpenAI API key:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import os\n",
|
||||
"\n",
|
||||
"os.environ[\"OPENAI_API_KEY\"] = \"Your OpenAI API key\"\n",
|
||||
"os.environ[\"APIFY_API_TOKEN\"] = \"Your Apify API token\"\n",
|
||||
"\n",
|
||||
"apify = ApifyWrapper()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Then run the Actor, wait for it to finish, and fetch its results from the Apify dataset into a LangChain document loader.\n",
|
||||
"\n",
|
||||
"Note that if you already have some results in an Apify dataset, you can load them directly using `ApifyDatasetLoader`, as shown in [this notebook](/docs/modules/data_connection/document_loaders/integrations/apify_dataset.html). In that notebook, you'll also find the explanation of the `dataset_mapping_function`, which is used to map fields from the Apify dataset records to LangChain `Document` fields."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"loader = apify.call_actor(\n",
|
||||
" actor_id=\"apify/website-content-crawler\",\n",
|
||||
" run_input={\"startUrls\": [{\"url\": \"https://python.langchain.com/en/latest/\"}]},\n",
|
||||
" dataset_mapping_function=lambda item: Document(\n",
|
||||
" page_content=item[\"text\"] or \"\", metadata={\"source\": item[\"url\"]}\n",
|
||||
" ),\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Initialize the vector index from the crawled documents:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"index = VectorstoreIndexCreator().from_loaders([loader])"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"And finally, query the vector index:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"query = \"What is LangChain?\"\n",
|
||||
"result = index.query_with_sources(query)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" LangChain is a standard interface through which you can interact with a variety of large language models (LLMs). It provides modules that can be used to build language model applications, and it also provides chains and agents with memory capabilities.\n",
|
||||
"\n",
|
||||
"https://python.langchain.com/en/latest/modules/models/llms.html, https://python.langchain.com/en/latest/getting_started/getting_started.html\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"print(result[\"answer\"])\n",
|
||||
"print(result[\"sources\"])"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3 (ipykernel)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.11.3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
}
|
||||
@ -1,19 +0,0 @@
|
||||
from langchain.experimental.autonomous_agents.autogpt.agent import AutoGPT
|
||||
from langchain.experimental.autonomous_agents.baby_agi.baby_agi import BabyAGI
|
||||
from langchain.experimental.generative_agents.generative_agent import GenerativeAgent
|
||||
from langchain.experimental.generative_agents.memory import GenerativeAgentMemory
|
||||
from langchain.experimental.plan_and_execute import (
|
||||
PlanAndExecute,
|
||||
load_agent_executor,
|
||||
load_chat_planner,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"BabyAGI",
|
||||
"AutoGPT",
|
||||
"GenerativeAgent",
|
||||
"GenerativeAgentMemory",
|
||||
"PlanAndExecute",
|
||||
"load_agent_executor",
|
||||
"load_chat_planner",
|
||||
]
|
||||
@ -1,4 +0,0 @@
|
||||
from langchain.experimental.autonomous_agents.autogpt.agent import AutoGPT
|
||||
from langchain.experimental.autonomous_agents.baby_agi.baby_agi import BabyAGI
|
||||
|
||||
__all__ = ["BabyAGI", "AutoGPT"]
|
||||
@ -1,6 +0,0 @@
|
||||
"""Experimental LLM wrappers."""
|
||||
|
||||
from langchain.experimental.llms.jsonformer_decoder import JsonFormer
|
||||
from langchain.experimental.llms.rellm_decoder import RELLM
|
||||
|
||||
__all__ = ["RELLM", "JsonFormer"]
|
||||
@ -1,9 +0,0 @@
|
||||
from langchain.experimental.plan_and_execute.agent_executor import PlanAndExecute
|
||||
from langchain.experimental.plan_and_execute.executors.agent_executor import (
|
||||
load_agent_executor,
|
||||
)
|
||||
from langchain.experimental.plan_and_execute.planners.chat_planner import (
|
||||
load_chat_planner,
|
||||
)
|
||||
|
||||
__all__ = ["PlanAndExecute", "load_agent_executor", "load_chat_planner"]
|
||||
@ -1,3 +0,0 @@
|
||||
from langchain.experimental.prompts.load import load_prompt
|
||||
|
||||
__all__ = ["load_prompt"]
|
||||
@ -0,0 +1,19 @@
|
||||
from langchain_experimental.autonomous_agents.autogpt.agent import AutoGPT
|
||||
from langchain_experimental.autonomous_agents.baby_agi.baby_agi import BabyAGI
|
||||
from langchain_experimental.generative_agents.generative_agent import GenerativeAgent
|
||||
from langchain_experimental.generative_agents.memory import GenerativeAgentMemory
|
||||
from langchain_experimental.plan_and_execute import (
|
||||
PlanAndExecute,
|
||||
load_agent_executor,
|
||||
load_chat_planner,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"BabyAGI",
|
||||
"AutoGPT",
|
||||
"GenerativeAgent",
|
||||
"GenerativeAgentMemory",
|
||||
"PlanAndExecute",
|
||||
"load_agent_executor",
|
||||
"load_chat_planner",
|
||||
]
|
||||
@ -0,0 +1,4 @@
|
||||
from langchain_experimental.autonomous_agents.autogpt.agent import AutoGPT
|
||||
from langchain_experimental.autonomous_agents.baby_agi.baby_agi import BabyAGI
|
||||
|
||||
__all__ = ["BabyAGI", "AutoGPT"]
|
||||
@ -1,9 +1,8 @@
|
||||
from typing import Any, Dict, List
|
||||
|
||||
from pydantic import Field
|
||||
|
||||
from langchain.memory.chat_memory import BaseChatMemory, get_prompt_input_key
|
||||
from langchain.vectorstores.base import VectorStoreRetriever
|
||||
from pydantic import Field
|
||||
|
||||
|
||||
class AutoGPTMemory(BaseChatMemory):
|
||||
@ -1,15 +1,15 @@
|
||||
import time
|
||||
from typing import Any, Callable, List
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
from langchain.experimental.autonomous_agents.autogpt.prompt_generator import get_prompt
|
||||
from langchain.prompts.chat import (
|
||||
BaseChatPromptTemplate,
|
||||
)
|
||||
from langchain.schema.messages import BaseMessage, HumanMessage, SystemMessage
|
||||
from langchain.tools.base import BaseTool
|
||||
from langchain.vectorstores.base import VectorStoreRetriever
|
||||
from pydantic import BaseModel
|
||||
|
||||
from langchain_experimental.autonomous_agents.autogpt.prompt_generator import get_prompt
|
||||
|
||||
|
||||
class AutoGPTPrompt(BaseChatPromptTemplate, BaseModel):
|
||||
@ -1,11 +1,11 @@
|
||||
from langchain.experimental.autonomous_agents.baby_agi.baby_agi import BabyAGI
|
||||
from langchain.experimental.autonomous_agents.baby_agi.task_creation import (
|
||||
from langchain_experimental.autonomous_agents.baby_agi.baby_agi import BabyAGI
|
||||
from langchain_experimental.autonomous_agents.baby_agi.task_creation import (
|
||||
TaskCreationChain,
|
||||
)
|
||||
from langchain.experimental.autonomous_agents.baby_agi.task_execution import (
|
||||
from langchain_experimental.autonomous_agents.baby_agi.task_execution import (
|
||||
TaskExecutionChain,
|
||||
)
|
||||
from langchain.experimental.autonomous_agents.baby_agi.task_prioritization import (
|
||||
from langchain_experimental.autonomous_agents.baby_agi.task_prioritization import (
|
||||
TaskPrioritizationChain,
|
||||
)
|
||||
|
||||
@ -1,5 +1,5 @@
|
||||
"""Generative Agents primitives."""
|
||||
from langchain.experimental.generative_agents.generative_agent import GenerativeAgent
|
||||
from langchain.experimental.generative_agents.memory import GenerativeAgentMemory
|
||||
from langchain_experimental.generative_agents.generative_agent import GenerativeAgent
|
||||
from langchain_experimental.generative_agents.memory import GenerativeAgentMemory
|
||||
|
||||
__all__ = ["GenerativeAgent", "GenerativeAgentMemory"]
|
||||
@ -0,0 +1,6 @@
|
||||
"""Experimental LLM wrappers."""
|
||||
|
||||
from langchain_experimental.llms.jsonformer_decoder import JsonFormer
|
||||
from langchain_experimental.llms.rellm_decoder import RELLM
|
||||
|
||||
__all__ = ["RELLM", "JsonFormer"]
|
||||
@ -0,0 +1,9 @@
|
||||
from langchain_experimental.plan_and_execute.agent_executor import PlanAndExecute
|
||||
from langchain_experimental.plan_and_execute.executors.agent_executor import (
|
||||
load_agent_executor,
|
||||
)
|
||||
from langchain_experimental.plan_and_execute.planners.chat_planner import (
|
||||
load_chat_planner,
|
||||
)
|
||||
|
||||
__all__ = ["PlanAndExecute", "load_agent_executor", "load_chat_planner"]
|
||||
@ -1,12 +1,12 @@
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
from pydantic import Field
|
||||
|
||||
from langchain.callbacks.manager import CallbackManagerForChainRun
|
||||
from langchain.chains.base import Chain
|
||||
from langchain.experimental.plan_and_execute.executors.base import BaseExecutor
|
||||
from langchain.experimental.plan_and_execute.planners.base import BasePlanner
|
||||
from langchain.experimental.plan_and_execute.schema import (
|
||||
from pydantic import Field
|
||||
|
||||
from langchain_experimental.plan_and_execute.executors.base import BaseExecutor
|
||||
from langchain_experimental.plan_and_execute.planners.base import BasePlanner
|
||||
from langchain_experimental.plan_and_execute.schema import (
|
||||
BaseStepContainer,
|
||||
ListStepContainer,
|
||||
)
|
||||
@ -1,11 +1,11 @@
|
||||
from abc import abstractmethod
|
||||
from typing import Any
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
from langchain.callbacks.manager import Callbacks
|
||||
from langchain.chains.base import Chain
|
||||
from langchain.experimental.plan_and_execute.schema import StepResponse
|
||||
from pydantic import BaseModel
|
||||
|
||||
from langchain_experimental.plan_and_execute.schema import StepResponse
|
||||
|
||||
|
||||
class BaseExecutor(BaseModel):
|
||||
@ -1,11 +1,11 @@
|
||||
from abc import abstractmethod
|
||||
from typing import Any, List, Optional
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
from langchain.callbacks.manager import Callbacks
|
||||
from langchain.chains.llm import LLMChain
|
||||
from langchain.experimental.plan_and_execute.schema import Plan, PlanOutputParser
|
||||
from pydantic import BaseModel
|
||||
|
||||
from langchain_experimental.plan_and_execute.schema import Plan, PlanOutputParser
|
||||
|
||||
|
||||
class BasePlanner(BaseModel):
|
||||
@ -1,15 +1,16 @@
|
||||
import re
|
||||
|
||||
from langchain.chains import LLMChain
|
||||
from langchain.experimental.plan_and_execute.planners.base import LLMPlanner
|
||||
from langchain.experimental.plan_and_execute.schema import (
|
||||
from langchain.prompts import ChatPromptTemplate, HumanMessagePromptTemplate
|
||||
from langchain.schema.language_model import BaseLanguageModel
|
||||
from langchain.schema.messages import SystemMessage
|
||||
|
||||
from langchain_experimental.plan_and_execute.planners.base import LLMPlanner
|
||||
from langchain_experimental.plan_and_execute.schema import (
|
||||
Plan,
|
||||
PlanOutputParser,
|
||||
Step,
|
||||
)
|
||||
from langchain.prompts import ChatPromptTemplate, HumanMessagePromptTemplate
|
||||
from langchain.schema.language_model import BaseLanguageModel
|
||||
from langchain.schema.messages import SystemMessage
|
||||
|
||||
SYSTEM_PROMPT = (
|
||||
"Let's first understand the problem and devise a plan to solve the problem."
|
||||
@ -1,9 +1,8 @@
|
||||
from abc import abstractmethod
|
||||
from typing import List, Tuple
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from langchain.schema import BaseOutputParser
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
|
||||
class Step(BaseModel):
|
||||
@ -0,0 +1,3 @@
|
||||
from langchain_experimental.prompts.load import load_prompt
|
||||
|
||||
__all__ = ["load_prompt"]
|
||||
@ -1,4 +1,4 @@
|
||||
"""Chain for interacting with SQL Database."""
|
||||
from langchain.experimental.sql.base import SQLDatabaseChain
|
||||
from langchain_experimental.sql.base import SQLDatabaseChain
|
||||
|
||||
__all__ = ["SQLDatabaseChain"]
|
||||
@ -1,205 +0,0 @@
|
||||
from typing import Any, Callable, Dict, Optional
|
||||
|
||||
from pydantic import BaseModel, root_validator
|
||||
|
||||
from langchain.document_loaders import ApifyDatasetLoader
|
||||
from langchain.document_loaders.base import Document
|
||||
from langchain.utils import get_from_dict_or_env
|
||||
|
||||
|
||||
class ApifyWrapper(BaseModel):
|
||||
"""Wrapper around Apify.
|
||||
|
||||
To use, you should have the ``apify-client`` python package installed,
|
||||
and the environment variable ``APIFY_API_TOKEN`` set with your API key, or pass
|
||||
`apify_api_token` as a named parameter to the constructor.
|
||||
"""
|
||||
|
||||
apify_client: Any
|
||||
apify_client_async: Any
|
||||
|
||||
@root_validator()
|
||||
def validate_environment(cls, values: Dict) -> Dict:
|
||||
"""Validate environment.
|
||||
|
||||
Validate that an Apify API token is set and the apify-client
|
||||
Python package exists in the current environment.
|
||||
"""
|
||||
apify_api_token = get_from_dict_or_env(
|
||||
values, "apify_api_token", "APIFY_API_TOKEN"
|
||||
)
|
||||
|
||||
try:
|
||||
from apify_client import ApifyClient, ApifyClientAsync
|
||||
|
||||
values["apify_client"] = ApifyClient(apify_api_token)
|
||||
values["apify_client_async"] = ApifyClientAsync(apify_api_token)
|
||||
except ImportError:
|
||||
raise ValueError(
|
||||
"Could not import apify-client Python package. "
|
||||
"Please install it with `pip install apify-client`."
|
||||
)
|
||||
|
||||
return values
|
||||
|
||||
def call_actor(
|
||||
self,
|
||||
actor_id: str,
|
||||
run_input: Dict,
|
||||
dataset_mapping_function: Callable[[Dict], Document],
|
||||
*,
|
||||
build: Optional[str] = None,
|
||||
memory_mbytes: Optional[int] = None,
|
||||
timeout_secs: Optional[int] = None,
|
||||
) -> ApifyDatasetLoader:
|
||||
"""Run an Actor on the Apify platform and wait for results to be ready.
|
||||
|
||||
Args:
|
||||
actor_id (str): The ID or name of the Actor on the Apify platform.
|
||||
run_input (Dict): The input object of the Actor that you're trying to run.
|
||||
dataset_mapping_function (Callable): A function that takes a single
|
||||
dictionary (an Apify dataset item) and converts it to an
|
||||
instance of the Document class.
|
||||
build (str, optional): Optionally specifies the actor build to run.
|
||||
It can be either a build tag or build number.
|
||||
memory_mbytes (int, optional): Optional memory limit for the run,
|
||||
in megabytes.
|
||||
timeout_secs (int, optional): Optional timeout for the run, in seconds.
|
||||
|
||||
Returns:
|
||||
ApifyDatasetLoader: A loader that will fetch the records from the
|
||||
Actor run's default dataset.
|
||||
"""
|
||||
actor_call = self.apify_client.actor(actor_id).call(
|
||||
run_input=run_input,
|
||||
build=build,
|
||||
memory_mbytes=memory_mbytes,
|
||||
timeout_secs=timeout_secs,
|
||||
)
|
||||
|
||||
return ApifyDatasetLoader(
|
||||
dataset_id=actor_call["defaultDatasetId"],
|
||||
dataset_mapping_function=dataset_mapping_function,
|
||||
)
|
||||
|
||||
async def acall_actor(
|
||||
self,
|
||||
actor_id: str,
|
||||
run_input: Dict,
|
||||
dataset_mapping_function: Callable[[Dict], Document],
|
||||
*,
|
||||
build: Optional[str] = None,
|
||||
memory_mbytes: Optional[int] = None,
|
||||
timeout_secs: Optional[int] = None,
|
||||
) -> ApifyDatasetLoader:
|
||||
"""Run an Actor on the Apify platform and wait for results to be ready.
|
||||
|
||||
Args:
|
||||
actor_id (str): The ID or name of the Actor on the Apify platform.
|
||||
run_input (Dict): The input object of the Actor that you're trying to run.
|
||||
dataset_mapping_function (Callable): A function that takes a single
|
||||
dictionary (an Apify dataset item) and converts it to
|
||||
an instance of the Document class.
|
||||
build (str, optional): Optionally specifies the actor build to run.
|
||||
It can be either a build tag or build number.
|
||||
memory_mbytes (int, optional): Optional memory limit for the run,
|
||||
in megabytes.
|
||||
timeout_secs (int, optional): Optional timeout for the run, in seconds.
|
||||
|
||||
Returns:
|
||||
ApifyDatasetLoader: A loader that will fetch the records from the
|
||||
Actor run's default dataset.
|
||||
"""
|
||||
actor_call = await self.apify_client_async.actor(actor_id).call(
|
||||
run_input=run_input,
|
||||
build=build,
|
||||
memory_mbytes=memory_mbytes,
|
||||
timeout_secs=timeout_secs,
|
||||
)
|
||||
|
||||
return ApifyDatasetLoader(
|
||||
dataset_id=actor_call["defaultDatasetId"],
|
||||
dataset_mapping_function=dataset_mapping_function,
|
||||
)
|
||||
|
||||
def call_actor_task(
|
||||
self,
|
||||
task_id: str,
|
||||
task_input: Dict,
|
||||
dataset_mapping_function: Callable[[Dict], Document],
|
||||
*,
|
||||
build: Optional[str] = None,
|
||||
memory_mbytes: Optional[int] = None,
|
||||
timeout_secs: Optional[int] = None,
|
||||
) -> ApifyDatasetLoader:
|
||||
"""Run a saved Actor task on Apify and wait for results to be ready.
|
||||
|
||||
Args:
|
||||
task_id (str): The ID or name of the task on the Apify platform.
|
||||
task_input (Dict): The input object of the task that you're trying to run.
|
||||
Overrides the task's saved input.
|
||||
dataset_mapping_function (Callable): A function that takes a single
|
||||
dictionary (an Apify dataset item) and converts it to an
|
||||
instance of the Document class.
|
||||
build (str, optional): Optionally specifies the actor build to run.
|
||||
It can be either a build tag or build number.
|
||||
memory_mbytes (int, optional): Optional memory limit for the run,
|
||||
in megabytes.
|
||||
timeout_secs (int, optional): Optional timeout for the run, in seconds.
|
||||
|
||||
Returns:
|
||||
ApifyDatasetLoader: A loader that will fetch the records from the
|
||||
task run's default dataset.
|
||||
"""
|
||||
task_call = self.apify_client.task(task_id).call(
|
||||
task_input=task_input,
|
||||
build=build,
|
||||
memory_mbytes=memory_mbytes,
|
||||
timeout_secs=timeout_secs,
|
||||
)
|
||||
|
||||
return ApifyDatasetLoader(
|
||||
dataset_id=task_call["defaultDatasetId"],
|
||||
dataset_mapping_function=dataset_mapping_function,
|
||||
)
|
||||
|
||||
async def acall_actor_task(
|
||||
self,
|
||||
task_id: str,
|
||||
task_input: Dict,
|
||||
dataset_mapping_function: Callable[[Dict], Document],
|
||||
*,
|
||||
build: Optional[str] = None,
|
||||
memory_mbytes: Optional[int] = None,
|
||||
timeout_secs: Optional[int] = None,
|
||||
) -> ApifyDatasetLoader:
|
||||
"""Run a saved Actor task on Apify and wait for results to be ready.
|
||||
|
||||
Args:
|
||||
task_id (str): The ID or name of the task on the Apify platform.
|
||||
task_input (Dict): The input object of the task that you're trying to run.
|
||||
Overrides the task's saved input.
|
||||
dataset_mapping_function (Callable): A function that takes a single
|
||||
dictionary (an Apify dataset item) and converts it to an
|
||||
instance of the Document class.
|
||||
build (str, optional): Optionally specifies the actor build to run.
|
||||
It can be either a build tag or build number.
|
||||
memory_mbytes (int, optional): Optional memory limit for the run,
|
||||
in megabytes.
|
||||
timeout_secs (int, optional): Optional timeout for the run, in seconds.
|
||||
|
||||
Returns:
|
||||
ApifyDatasetLoader: A loader that will fetch the records from the
|
||||
task run's default dataset.
|
||||
"""
|
||||
task_call = await self.apify_client_async.task(task_id).call(
|
||||
task_input=task_input,
|
||||
build=build,
|
||||
memory_mbytes=memory_mbytes,
|
||||
timeout_secs=timeout_secs,
|
||||
)
|
||||
|
||||
return ApifyDatasetLoader(
|
||||
dataset_id=task_call["defaultDatasetId"],
|
||||
dataset_mapping_function=dataset_mapping_function,
|
||||
)
|
||||
Loading…
Reference in New Issue