반응형

사내에서 Langchain을 사용할 경우 아래와 같이 오류가 발생한다면 모듈 파일을 수정해서 오류를 우회할 수 있습니다.

 

코드

llm = ChatOpenAI(
    api_key="USER API Key",
    model="gpt-4o-mini" 
)

@tool
def get_weather(city: str) -> str:
    """Get weather for a given city.""" 
    return f"It's always sunny in {city}!" 

agent = create_agent(
    model=llm,
    tools=[get_weather],
    system_prompt="You are a helpful assistant",
)

# Run the agent
agent.invoke(
    {"messages": [{"role": "user", "content": "what is the weather in sf"}]}
)

 

✅ 오류 내용

---------------------------------------------------------------------------
ConnectError                              Traceback (most recent call last)
File c:\Users\User\.conda\envs\Orange\Lib\site-packages\httpx\_transports\default.py:101, in map_httpcore_exceptions()
    100 try:
--> 101     yield
    102 except Exception as exc:

File c:\Users\User\.conda\envs\Orange\Lib\site-packages\httpx\_transports\default.py:250, in HTTPTransport.handle_request(self, request)
    249 with map_httpcore_exceptions():
--> 250     resp = self._pool.handle_request(req)
    252 assert isinstance(resp.stream, typing.Iterable)

File c:\Users\User\.conda\envs\Orange\Lib\site-packages\httpcore\_sync\connection_pool.py:256, in ConnectionPool.handle_request(self, request)
    255     self._close_connections(closing)
--> 256     raise exc from None
    258 # Return the response. Note that in this case we still have to manage
    259 # the point at which the response is closed.

File c:\Users\User\.conda\envs\Orange\Lib\site-packages\httpcore\_sync\connection_pool.py:236, in ConnectionPool.handle_request(self, request)
    234 try:
    235     # Send the request on the assigned connection.
--> 236     response = connection.handle_request(
    237         pool_request.request
    238     )
    239 except ConnectionNotAvailable:
    240     # In some cases a connection may initially be available to
    241     # handle a request, but then become unavailable.
    242     #
    243     # In this case we clear the connection and try again.

File c:\Users\User\.conda\envs\Orange\Lib\site-packages\httpcore\_sync\connection.py:101, in HTTPConnection.handle_request(self, request)
    100     self._connect_failed = True
--> 101     raise exc
    103 return self._connection.handle_request(request)

File c:\Users\User\.conda\envs\Orange\Lib\site-packages\httpcore\_sync\connection.py:78, in HTTPConnection.handle_request(self, request)
     77 if self._connection is None:
---> 78     stream = self._connect(request)
     80     ssl_object = stream.get_extra_info("ssl_object")

File c:\Users\User\.conda\envs\Orange\Lib\site-packages\httpcore\_sync\connection.py:156, in HTTPConnection._connect(self, request)
    155 with Trace("start_tls", logger, request, kwargs) as trace:
--> 156     stream = stream.start_tls(**kwargs)
    157     trace.return_value = stream

File c:\Users\User\.conda\envs\Orange\Lib\site-packages\httpcore\_backends\sync.py:154, in SyncStream.start_tls(self, ssl_context, server_hostname, timeout)
    150 exc_map: ExceptionMapping = {
    151     socket.timeout: ConnectTimeout,
    152     OSError: ConnectError,
    153 }
--> 154 with map_exceptions(exc_map):
    155     try:

File c:\Users\User\.conda\envs\Orange\Lib\contextlib.py:158, in _GeneratorContextManager.__exit__(self, typ, value, traceback)
    157 try:
--> 158     self.gen.throw(value)
    159 except StopIteration as exc:
    160     # Suppress StopIteration *unless* it's the same exception that
    161     # was passed to throw().  This prevents a StopIteration
    162     # raised inside the "with" statement from being suppressed.

File c:\Users\User\.conda\envs\Orange\Lib\site-packages\httpcore\_exceptions.py:14, in map_exceptions(map)
     13     if isinstance(exc, from_exc):
---> 14         raise to_exc(exc) from exc
     15 raise

ConnectError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1000)

The above exception was the direct cause of the following exception:

ConnectError                              Traceback (most recent call last)
File c:\Users\User\.conda\envs\Orange\Lib\site-packages\openai\_base_client.py:982, in SyncAPIClient.request(self, cast_to, options, stream, stream_cls)
    981 try:
--> 982     response = self._client.send(
    983         request,
    984         stream=stream or self._should_stream_response_body(request=request),
    985         **kwargs,
    986     )
    987 except httpx.TimeoutException as err:

File c:\Users\User\.conda\envs\Orange\Lib\site-packages\httpx\_client.py:914, in Client.send(self, request, stream, auth, follow_redirects)
    912 auth = self._build_request_auth(request, auth)
--> 914 response = self._send_handling_auth(
    915     request,
    916     auth=auth,
    917     follow_redirects=follow_redirects,
    918     history=[],
    919 )
    920 try:

File c:\Users\User\.conda\envs\Orange\Lib\site-packages\httpx\_client.py:942, in Client._send_handling_auth(self, request, auth, follow_redirects, history)
    941 while True:
--> 942     response = self._send_handling_redirects(
    943         request,
    944         follow_redirects=follow_redirects,
    945         history=history,
    946     )
    947     try:

File c:\Users\User\.conda\envs\Orange\Lib\site-packages\httpx\_client.py:979, in Client._send_handling_redirects(self, request, follow_redirects, history)
    977     hook(request)
--> 979 response = self._send_single_request(request)
    980 try:

File c:\Users\User\.conda\envs\Orange\Lib\site-packages\httpx\_client.py:1014, in Client._send_single_request(self, request)
   1013 with request_context(request=request):
-> 1014     response = transport.handle_request(request)
   1016 assert isinstance(response.stream, SyncByteStream)

File c:\Users\User\.conda\envs\Orange\Lib\site-packages\httpx\_transports\default.py:249, in HTTPTransport.handle_request(self, request)
    237 req = httpcore.Request(
    238     method=request.method,
    239     url=httpcore.URL(
   (...)    247     extensions=request.extensions,
    248 )
--> 249 with map_httpcore_exceptions():
    250     resp = self._pool.handle_request(req)

File c:\Users\User\.conda\envs\Orange\Lib\contextlib.py:158, in _GeneratorContextManager.__exit__(self, typ, value, traceback)
    157 try:
--> 158     self.gen.throw(value)
    159 except StopIteration as exc:
    160     # Suppress StopIteration *unless* it's the same exception that
    161     # was passed to throw().  This prevents a StopIteration
    162     # raised inside the "with" statement from being suppressed.

File c:\Users\User\.conda\envs\Orange\Lib\site-packages\httpx\_transports\default.py:118, in map_httpcore_exceptions()
    117 message = str(exc)
--> 118 raise mapped_exc(message) from exc

ConnectError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1000)

The above exception was the direct cause of the following exception:

APIConnectionError                        Traceback (most recent call last)
Cell In[3], line 17
     10 agent = create_agent(
     11     model=llm,
     12     tools=[get_weather],
     13     system_prompt="You are a helpful assistant",
     14 )
     16 # Run the agent
---> 17 agent.invoke(
     18     {"messages": [{"role": "user", "content": "what is the weather in sf"}]}
     19 )

File c:\Users\User\.conda\envs\Orange\Lib\site-packages\langgraph\pregel\main.py:3094, in Pregel.invoke(self, input, config, context, stream_mode, print_mode, output_keys, interrupt_before, interrupt_after, durability, **kwargs)
   3091 chunks: list[dict[str, Any] | Any] = []
   3092 interrupts: list[Interrupt] = []
-> 3094 for chunk in self.stream(
   3095     input,
   3096     config,
   3097     context=context,
   3098     stream_mode=["updates", "values"]
   3099     if stream_mode == "values" 
   3100     else stream_mode,
   3101     print_mode=print_mode,
   3102     output_keys=output_keys,
   3103     interrupt_before=interrupt_before,
   3104     interrupt_after=interrupt_after,
   3105     durability=durability,
   3106     **kwargs,
   3107 ):
   3108     if stream_mode == "values":
   3109         if len(chunk) == 2:

File c:\Users\User\.conda\envs\Orange\Lib\site-packages\langgraph\pregel\main.py:2679, in Pregel.stream(self, input, config, context, stream_mode, print_mode, output_keys, interrupt_before, interrupt_after, durability, subgraphs, debug, **kwargs)
   2677 for task in loop.match_cached_writes():
   2678     loop.output_writes(task.id, task.writes, cached=True)
-> 2679 for _ in runner.tick(
   2680     [t for t in loop.tasks.values() if not t.writes],
   2681     timeout=self.step_timeout,
   2682     get_waiter=get_waiter,
   2683     schedule_task=loop.accept_push,
   2684 ):
   2685     # emit output
   2686     yield from _output(
   2687         stream_mode, print_mode, subgraphs, stream.get, queue.Empty
   2688     )
   2689 loop.after_tick()

File c:\Users\User\.conda\envs\Orange\Lib\site-packages\langgraph\pregel\_runner.py:167, in PregelRunner.tick(self, tasks, reraise, timeout, retry_policy, get_waiter, schedule_task)
    165 t = tasks[0]
    166 try:
--> 167     run_with_retry(
    168         t,
    169         retry_policy,
    170         configurable={
    171             CONFIG_KEY_CALL: partial(
    172                 _call,
    173                 weakref.ref(t),
    174                 retry_policy=retry_policy,
    175                 futures=weakref.ref(futures),
    176                 schedule_task=schedule_task,
    177                 submit=self.submit,
    178             ),
    179         },
    180     )
    181     self.commit(t, None)
    182 except Exception as exc:

File c:\Users\User\.conda\envs\Orange\Lib\site-packages\langgraph\pregel\_retry.py:42, in run_with_retry(task, retry_policy, configurable)
     40     task.writes.clear()
     41     # run the task
---> 42     return task.proc.invoke(task.input, config)
     43 except ParentCommand as exc:
     44     ns: str = config[CONF][CONFIG_KEY_CHECKPOINT_NS]

File c:\Users\User\.conda\envs\Orange\Lib\site-packages\langgraph\_internal\_runnable.py:656, in RunnableSeq.invoke(self, input, config, **kwargs)
    654     # run in context
    655     with set_config_context(config, run) as context:
--> 656         input = context.run(step.invoke, input, config, **kwargs)
    657 else:
    658     input = step.invoke(input, config)

File c:\Users\User\.conda\envs\Orange\Lib\site-packages\langgraph\_internal\_runnable.py:400, in RunnableCallable.invoke(self, input, config, **kwargs)
    398         run_manager.on_chain_end(ret)
    399 else:
--> 400     ret = self.func(*args, **kwargs)
    401 if self.recurse and isinstance(ret, Runnable):
    402     return ret.invoke(input, config)

File c:\Users\User\.conda\envs\Orange\Lib\site-packages\langchain\agents\factory.py:1034, in create_agent.<locals>.model_node(state, runtime)
   1021 request = ModelRequest(
   1022     model=model,
   1023     tools=default_tools,
   (...)   1029     runtime=runtime,
   1030 )
   1032 if wrap_model_call_handler is None:
   1033     # No handlers - execute directly
-> 1034     response = _execute_model_sync(request)
   1035 else:
   1036     # Call composed handler with base handler
   1037     response = wrap_model_call_handler(request, _execute_model_sync)

File c:\Users\User\.conda\envs\Orange\Lib\site-packages\langchain\agents\factory.py:1007, in create_agent.<locals>._execute_model_sync(request)
   1004 if request.system_prompt:
   1005     messages = [SystemMessage(request.system_prompt), *messages]
-> 1007 output = model_.invoke(messages)
   1009 # Handle model output to get messages and structured_response
   1010 handled_output = _handle_model_output(output, effective_response_format)

File c:\Users\User\.conda\envs\Orange\Lib\site-packages\langchain_core\runnables\base.py:5489, in RunnableBindingBase.invoke(self, input, config, **kwargs)
   5482 @override
   5483 def invoke(
   5484     self,
   (...)   5487     **kwargs: Any | None,
   5488 ) -> Output:
-> 5489     return self.bound.invoke(
   5490         input,
   5491         self._merge_configs(config),
   5492         **{**self.kwargs, **kwargs},
   5493     )

File c:\Users\User\.conda\envs\Orange\Lib\site-packages\langchain_core\language_models\chat_models.py:379, in BaseChatModel.invoke(self, input, config, stop, **kwargs)
    365 @override
    366 def invoke(
    367     self,
   (...)    372     **kwargs: Any,
    373 ) -> AIMessage:
    374     config = ensure_config(config)
    375     return cast(
    376         "AIMessage",
    377         cast(
    378             "ChatGeneration",
--> 379             self.generate_prompt(
    380                 [self._convert_input(input)],
    381                 stop=stop,
    382                 callbacks=config.get("callbacks"),
    383                 tags=config.get("tags"),
    384                 metadata=config.get("metadata"),
    385                 run_name=config.get("run_name"),
    386                 run_id=config.pop("run_id", None),
    387                 **kwargs,
    388             ).generations[0][0],
    389         ).message,
    390     )

File c:\Users\User\.conda\envs\Orange\Lib\site-packages\langchain_core\language_models\chat_models.py:1088, in BaseChatModel.generate_prompt(self, prompts, stop, callbacks, **kwargs)
   1079 @override
   1080 def generate_prompt(
   1081     self,
   (...)   1085     **kwargs: Any,
   1086 ) -> LLMResult:
   1087     prompt_messages = [p.to_messages() for p in prompts]
-> 1088     return self.generate(prompt_messages, stop=stop, callbacks=callbacks, **kwargs)

File c:\Users\User\.conda\envs\Orange\Lib\site-packages\langchain_core\language_models\chat_models.py:903, in BaseChatModel.generate(self, messages, stop, callbacks, tags, metadata, run_name, run_id, **kwargs)
    900 for i, m in enumerate(input_messages):
    901     try:
    902         results.append(
--> 903             self._generate_with_cache(
    904                 m,
    905                 stop=stop,
    906                 run_manager=run_managers[i] if run_managers else None,
    907                 **kwargs,
    908             )
    909         )
    910     except BaseException as e:
    911         if run_managers:

File c:\Users\User\.conda\envs\Orange\Lib\site-packages\langchain_core\language_models\chat_models.py:1192, in BaseChatModel._generate_with_cache(self, messages, stop, run_manager, **kwargs)
   1190     result = generate_from_stream(iter(chunks))
   1191 elif inspect.signature(self._generate).parameters.get("run_manager"):
-> 1192     result = self._generate(
   1193         messages, stop=stop, run_manager=run_manager, **kwargs
   1194     )
   1195 else:
   1196     result = self._generate(messages, stop=stop, **kwargs)

File c:\Users\User\.conda\envs\Orange\Lib\site-packages\langchain_openai\chat_models\base.py:1300, in BaseChatOpenAI._generate(self, messages, stop, run_manager, **kwargs)
   1298     if raw_response is not None and hasattr(raw_response, "http_response"):
   1299         e.response = raw_response.http_response  # type: ignore[attr-defined]
-> 1300     raise e
   1301 if (
   1302     self.include_response_headers
   1303     and raw_response is not None
   1304     and hasattr(raw_response, "headers")
   1305 ):
   1306     generation_info = {"headers": dict(raw_response.headers)}

File c:\Users\User\.conda\envs\Orange\Lib\site-packages\langchain_openai\chat_models\base.py:1295, in BaseChatOpenAI._generate(self, messages, stop, run_manager, **kwargs)
   1288         return _construct_lc_result_from_responses_api(
   1289             response,
   1290             schema=original_schema_obj,
   1291             metadata=generation_info,
   1292             output_version=self.output_version,
   1293         )
   1294     else:
-> 1295         raw_response = self.client.with_raw_response.create(**payload)
   1296         response = raw_response.parse()
   1297 except Exception as e:

File c:\Users\User\.conda\envs\Orange\Lib\site-packages\openai\_legacy_response.py:364, in to_raw_response_wrapper.<locals>.wrapped(*args, **kwargs)
    360 extra_headers[RAW_RESPONSE_HEADER] = "true" 
    362 kwargs["extra_headers"] = extra_headers
--> 364 return cast(LegacyAPIResponse[R], func(*args, **kwargs))

File c:\Users\User\.conda\envs\Orange\Lib\site-packages\openai\_utils\_utils.py:286, in required_args.<locals>.inner.<locals>.wrapper(*args, **kwargs)
    284             msg = f"Missing required argument: {quote(missing[0])}" 
    285     raise TypeError(msg)
--> 286 return func(*args, **kwargs)

File c:\Users\User\.conda\envs\Orange\Lib\site-packages\openai\resources\chat\completions\completions.py:1156, in Completions.create(self, messages, model, audio, frequency_penalty, function_call, functions, logit_bias, logprobs, max_completion_tokens, max_tokens, metadata, modalities, n, parallel_tool_calls, prediction, presence_penalty, prompt_cache_key, reasoning_effort, response_format, safety_identifier, seed, service_tier, stop, store, stream, stream_options, temperature, tool_choice, tools, top_logprobs, top_p, user, verbosity, web_search_options, extra_headers, extra_query, extra_body, timeout)
   1110 @required_args(["messages", "model"], ["messages", "model", "stream"])
   1111 def create(
   1112     self,
   (...)   1153     timeout: float | httpx.Timeout | None | NotGiven = not_given,
   1154 ) -> ChatCompletion | Stream[ChatCompletionChunk]:
   1155     validate_response_format(response_format)
-> 1156     return self._post(
   1157         "/chat/completions",
   1158         body=maybe_transform(
   1159             {
   1160                 "messages": messages,
   1161                 "model": model,
   1162                 "audio": audio,
   1163                 "frequency_penalty": frequency_penalty,
   1164                 "function_call": function_call,
   1165                 "functions": functions,
   1166                 "logit_bias": logit_bias,
   1167                 "logprobs": logprobs,
   1168                 "max_completion_tokens": max_completion_tokens,
   1169                 "max_tokens": max_tokens,
   1170                 "metadata": metadata,
   1171                 "modalities": modalities,
   1172                 "n": n,
   1173                 "parallel_tool_calls": parallel_tool_calls,
   1174                 "prediction": prediction,
   1175                 "presence_penalty": presence_penalty,
   1176                 "prompt_cache_key": prompt_cache_key,
   1177                 "reasoning_effort": reasoning_effort,
   1178                 "response_format": response_format,
   1179                 "safety_identifier": safety_identifier,
   1180                 "seed": seed,
   1181                 "service_tier": service_tier,
   1182                 "stop": stop,
   1183                 "store": store,
   1184                 "stream": stream,
   1185                 "stream_options": stream_options,
   1186                 "temperature": temperature,
   1187                 "tool_choice": tool_choice,
   1188                 "tools": tools,
   1189                 "top_logprobs": top_logprobs,
   1190                 "top_p": top_p,
   1191                 "user": user,
   1192                 "verbosity": verbosity,
   1193                 "web_search_options": web_search_options,
   1194             },
   1195             completion_create_params.CompletionCreateParamsStreaming
   1196             if stream
   1197             else completion_create_params.CompletionCreateParamsNonStreaming,
   1198         ),
   1199         options=make_request_options(
   1200             extra_headers=extra_headers, extra_query=extra_query, extra_body=extra_body, timeout=timeout
   1201         ),
   1202         cast_to=ChatCompletion,
   1203         stream=stream or False,
   1204         stream_cls=Stream[ChatCompletionChunk],
   1205     )

File c:\Users\User\.conda\envs\Orange\Lib\site-packages\openai\_base_client.py:1259, in SyncAPIClient.post(self, path, cast_to, body, options, files, stream, stream_cls)
   1245 def post(
   1246     self,
   1247     path: str,
   (...)   1254     stream_cls: type[_StreamT] | None = None,
   1255 ) -> ResponseT | _StreamT:
   1256     opts = FinalRequestOptions.construct(
   1257         method="post", url=path, json_data=body, files=to_httpx_files(files), **options
   1258     )
-> 1259     return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))

File c:\Users\User\.conda\envs\Orange\Lib\site-packages\openai\_base_client.py:1014, in SyncAPIClient.request(self, cast_to, options, stream, stream_cls)
   1011         continue
   1013     log.debug("Raising connection error")
-> 1014     raise APIConnectionError(request=request) from err
   1016 log.debug(
   1017     'HTTP Response: %s %s "%i %s" %s',
   1018     request.method,
   (...)   1022     response.headers,
   1023 )
   1024 log.debug("request_id: %s", response.headers.get("x-request-id"))

APIConnectionError: Connection error.
During task with name 'model' and id '165fa95b-????-????-????-????209ef8c5'

 

☑️ 해결 방법

C:\Users\User\.conda\envs\Orange\Lib\site-packages\httpx\_transports\default.py

모듈이 설치된 폴더로 이동하여 default.py를 아래와 같이 수정합니다.

class HTTPTransport(BaseTransport):
    def __init__(
        self,
        verify: ssl.SSLContext | str | bool = True,
        cert: CertTypes | None = None,
        trust_env: bool = True,
        http1: bool = True,
        http2: bool = False,
        limits: Limits = DEFAULT_LIMITS,
        proxy: ProxyTypes | None = None,
        uds: str | None = None,
        local_address: str | None = None,
        retries: int = 0,
        socket_options: typing.Iterable[SOCKET_OPTION] | None = None,
    ) -> None:
        import httpcore
        verify=False    # TODO 강제 False설정
        proxy = Proxy(url=proxy) if isinstance(proxy, (str, URL)) else proxy
        ssl_context = create_ssl_context(verify=verify, cert=cert, trust_env=trust_env)

# ~~~~~~~~~~~~~~~~~~~~~~~~~~~ 생략 ~~~~~~~~~~~~~~~~~~~~~~~~~~~

class AsyncHTTPTransport(AsyncBaseTransport):
    def __init__(
        self,
        verify: ssl.SSLContext | str | bool = True,
        cert: CertTypes | None = None,
        trust_env: bool = True,
        http1: bool = True,
        http2: bool = False,
        limits: Limits = DEFAULT_LIMITS,
        proxy: ProxyTypes | None = None,
        uds: str | None = None,
        local_address: str | None = None,
        retries: int = 0,
        socket_options: typing.Iterable[SOCKET_OPTION] | None = None,
    ) -> None:
        import httpcore
        verify=False    # TODO 강제 False설정
        proxy = Proxy(url=proxy) if isinstance(proxy, (str, URL)) else proxy
        ssl_context = create_ssl_context(verify=verify, cert=cert, trust_env=trust_env)
        
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~ 생략 ~~~~~~~~~~~~~~~~~~~~~~~~~~~

 

반응형
반응형
  • ⭐PyTorch / TensorFlow
    ◦ 용도: 모델 훈련, 미분 계산, 텐서 연산
    ◦ [PyTorch]
        • 직관적이고 디버깅이 쉬워 LLM 연구와 실험에 널리 사용됨
        •  Hugging Face와의 연동도 뛰어남
    ◦ [TensorFlow]
        •   구글 생태계(GCP, TFX 등)와 잘 통합되며, 대규모 배포에 적합
        •   최근에는 PyTorch보다 덜 사용됨

 

  • Hugging Face ⭐Transformers
    ◦  용도: 사전학습된 LLM 모델 불러오기, fine-tuning, inference
        •  사전학습된 모델 (GPT, BERT, T5, LLaMA 등)을 쉽게 사용할 수 있게 해주는 라이브러리
        •  학습/추론에 필요한 tokenizer, 모델 구조, config 등을 자동으로 관리
        •  텍스트 생성, 분류, QA, 요약 등 다양한 작업에 최적화된 pipeline 제공
from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("gpt2")
model = AutoModelForCausalLM.from_pretrained("gpt2")

inputs = tokenizer("Hello, my name is", return_tensors="pt")
outputs = model.generate(**inputs)
print(tokenizer.decode(outputs[0]))

 

  • Datasets (Hugging Face Datasets)
    용도: 대규모 데이터셋 로딩, 전처리, 분할
        •   데이터셋을 빠르고 효율적으로 불러오고 사용할 수 있도록 지원
        •   대용량 파일도 메모리에 무리 없이 스트리밍 처리 가능
      Hugging Face Hub에서 바로 불러올 수 있음
from datasets import load_dataset

dataset = load_dataset("wikitext", "wikitext-2-raw-v1")
print(dataset["train"][0])

 

  • Accelerate / DeepSpeed / bitsandbytes
    ◦ 용도: 대규모 모델 훈련 시 하드웨어 최적화, 메모리 절약
        •    Accelerate: 다양한 하드웨어(CPU, GPU, TPU)에 맞게 모델 학습 코드 자동 조정
        •    DeepSpeed: 마이크로소프트에서 만든 초대형 모델 훈련 최적화 툴
        •    bitsandbytes: 8bit/4bit 양자화 학습 및 추론 지원 → 적은 메모리로 대형 모델 사용 가능

  • PEFT (Parameter-Efficient Fine-Tuning)
    ◦  용도: LLM을 빠르고 저렴하게 파인튜닝 (LoRA, Prefix Tuning 등)
        •    기존 LLM 전체가 아닌 일부 파라미터만 훈련해서 훨씬 적은 자원으로 fine-tuning 가능
        •    대표 기법: LoRA (Low-Rank Adaptation)
from peft import LoraConfig, get_peft_model

 

  • ⭐Tokenizers
    ◦  용도: LLM에 입력될 텍스트를 토큰 단위로 인코딩
        •    Hugging Face의 tokenizers는 빠르고 확장성 높은 토크나이저
        •    Byte-Pair Encoding (BPE), WordPiece, SentencePiece 등 다양한 방식 지원

  • ⭐LangChain / ⭐LlamaIndex
    ◦  용도: LLM 응용 프로그램 구축 (질의 응답, RAG, 에이전트 등)
        •    LangChain: 다양한 LLM, 데이터 소스, 도구를 연결해 에이전트처럼 동작하게 함
        •    LlamaIndex: 사용자 문서를 LLM에 연결해 QA 시스템 구축에 유용 (RAG)

  • WandB / TensorBoard
    ◦  용도: 모델 학습 과정 모니터링, 로그 기록, 하이퍼파라미터 튜닝
        •    실험 추적 및 시각화를 위한 필수 툴
        •    학습 정확도, 손실값, GPU 메모리 사용량 등 확인 가능

  • 부가적으로 알아두면 좋은 라이브러리
라이브러리 기능
scikit-learn 전통적인 ML 도구, 평가 지표 등
OpenLLM LLM 모델 배포에 특화된 프레임워크
fastapi API 서버를 빠르게 만들어 추론 시스템화
pandasnumpy 데이터 전처리 및 분석
torchtext 텍스트 데이터셋 처리 도구

 

  • 꼭 알아야 할 핵심 세트
범주 라이브러리
모델/훈련 PyTorchtransformers
데이터 datasetstokenizers
최적화 acceleratedeepspeedbitsandbytespeft
응용 LangChainLlamaIndex
실험 관리 wandbtensorboard

 

 

반응형

+ Recent posts