๋ณธ๋ฌธ

[์‹ค์Šต] ๋žญ์ฒด์ธ(LangChain) Entity Extraction ์˜ˆ์ œ - ๋ฆฌ๋ทฐ๊ฐ์ •๋ถ„์„GPT ๋งŒ๋“ค๊ธฐ

๋ฐ˜์‘ํ˜•

โœ…  ์‹ค์Šต ์ค€๋น„๋ฌผ: apikey

https://server-engineer.tistory.com/983

 

[์‹ค์Šต] OpenAI apikey ๋ฐœ๊ธ‰ ๋ฐ ์‚ฌ์šฉํ•˜๊ธฐ

1. ํšŒ์›๊ฐ€์ž…ํ•˜๊ธฐ ๋จผ์ € OpenAI ํ”Œ๋žซํผ์— ์ ‘์†ํ•ฉ๋‹ˆ๋‹ค. (๋ฐ”๋กœ๊ฐ€๊ธฐ) ํ™ˆํŽ˜์ด์ง€์—์„œ ํšŒ์›๊ฐ€์ž… ์ ˆ์ฐจ๋ฅผ ์™„๋ฃŒํ•ด์ฃผ์„ธ์š”. 2. API ํ‚ค ๋ฐœ๊ธ‰ ๋ฐ›๊ธฐ ๋กœ๊ทธ์ธ ํ›„, ์˜ค๋ฅธ์ชฝ ์ƒ๋‹จ์— ์žˆ๋Š” 'View API Keys'๋ฅผ ํด๋ฆญํ•ฉ๋‹ˆ๋‹ค. ์ง์ ‘ ๋ง

server-engineer.tistory.com


โœ…  ์‹ค์Šต ์ž๋ฃŒ

https://colab.research.google.com/github/i-am-shuan/learn-langchain/blob/main/langchain_entity_extraction_%EB%A6%AC%EB%B7%B0%EA%B0%90%EC%A0%95%EB%B6%84%EC%84%9DGPT.ipynb#scrollTo=pJ6i2qcdZ0Ow

 

langchain_entity_extraction_๋ฆฌ๋ทฐ๊ฐ์ •๋ถ„์„GPT.ipynb

Run, share, and edit Python notebooks

colab.research.google.com

 

 

โœ…  ์„œ๋น„์Šค ์„ค๋ช…

์„ค๋ช…:

๋ฆฌ๋ทฐ๊ฐ์ •๋ถ„์„GPT(SentimentGPT)๋Š” ์‚ฌ์šฉ์ž๊ฐ€ ์ž…๋ ฅํ•œ ๋ฆฌ๋ทฐ ํ…์ŠคํŠธ์˜ ๊ฐ์ •์„ ๋ถ„์„ํ•˜๊ณ , ๊ธ์ •, ๋ถ€์ •, ์ค‘๋ฆฝ์œผ๋กœ ๋ถ„๋ฅ˜ํ•˜๋Š” ์„œ๋น„์Šค์ž…๋‹ˆ๋‹ค. ์ด ์„œ๋น„์Šค๋Š” ํ•œ๊ตญ์–ด๋กœ ๋œ ๋ฆฌ๋ทฐ ํ…์ŠคํŠธ๋ฅผ ์ž…๋ ฅ๋ฐ›์•„ ๋ถ„์„ํ•ฉ๋‹ˆ๋‹ค.

 

์„œ๋น„์Šค ํ™œ์šฉ๋ฐฉ์•ˆ:

  • ๋ฆฌ๋ทฐ๊ฐ์ •๋ถ„์„GPT๋Š” ๋‹ค์Œ๊ณผ ๊ฐ™์€ ๋‹ค์–‘ํ•œ ๋ถ„์•ผ์—์„œ ํ™œ์šฉ๋  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
    • ๊ณ ๊ฐ ๋งŒ์กฑ๋„ ์กฐ์‚ฌ
    • ์ œํ’ˆ ๋ฆฌ๋ทฐ ๋ถ„์„
    • ์„œ๋น„์Šค ๊ฐœ์„ 
    • ๋งˆ์ผ€ํŒ… ์บ ํŽ˜์ธ ํšจ๊ณผ ๋ถ„์„

 

์‚ฌ์šฉ์ž ์งˆ๋ฌธ์—์„œ ์‘๋‹ต๊นŒ์ง€์˜ ๊ณผ์ •:

  1. ์‚ฌ์šฉ์ž๊ฐ€ ๋ฆฌ๋ทฐ ํ…์ŠคํŠธ๋ฅผ ์ž…๋ ฅํ•ฉ๋‹ˆ๋‹ค.
  2. ๋ฆฌ๋ทฐ๊ฐ์ •๋ถ„์„GPT๋Š” ์ž…๋ ฅ๋ฐ›์€ ๋ฆฌ๋ทฐ ํ…์ŠคํŠธ๋ฅผ ๋ถ„์„ํ•ฉ๋‹ˆ๋‹ค.
  3. ๋ฆฌ๋ทฐ๊ฐ์ •๋ถ„์„GPT๋Š” ๋ฆฌ๋ทฐ ํ…์ŠคํŠธ์˜ ๊ฐ์ •์„ ๊ธ์ •, ๋ถ€์ •, ์ค‘๋ฆฝ์œผ๋กœ ๋ถ„๋ฅ˜ํ•ฉ๋‹ˆ๋‹ค.
  4. ๋ฆฌ๋ทฐ๊ฐ์ •๋ถ„์„GPT๋Š” ์‚ฌ์šฉ์ž์—๊ฒŒ ๋ฆฌ๋ทฐ ํ…์ŠคํŠธ์˜ ๊ฐ์ •์„ ์•Œ๋ ค์ค๋‹ˆ๋‹ค.

 

์‚ฌ์šฉ๋œ ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ:

  • transformers: ์ž์—ฐ์–ด ์ฒ˜๋ฆฌ ์ž‘์—…์„ ์œ„ํ•œ ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ
  • torch: ๋”ฅ๋Ÿฌ๋‹ ํ”„๋ ˆ์ž„์›Œํฌ
  • numpy: ๊ณผํ•™ ๊ณ„์‚ฐ์„ ์œ„ํ•œ ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ
  • create_tagging_chain: ๋ฆฌ๋ทฐ ํ…์ŠคํŠธ์—์„œ ๊ธ์ •์ ์ธ ๋ถ€๋ถ„๊ณผ ๋ถ€์ •์ ์ธ ๋ถ€๋ถ„์„ ์ถ”์ถœํ•˜๋Š” ๋ฐ ์‚ฌ์šฉ

 

  • create_tagging_chain ์˜ˆ์‹œ
review_text = "์ด ์ œํ’ˆ์€ ์ •๋ง ์ข‹์•˜์Šต๋‹ˆ๋‹ค. ๋‹ค๋งŒ ๋ฐฐ์†ก์ด ์กฐ๊ธˆ ๋Š๋ ธ์Šต๋‹ˆ๋‹ค."

tagging_chain = create_tagging_chain(model, tokenizer, prompt)
positive_parts, negative_parts = tagging_chain(review_text)

print("๊ธ์ •์ ์ธ ๋ถ€๋ถ„:")
print(positive_parts)

print("๋ถ€์ •์ ์ธ ๋ถ€๋ถ„:")
print(negative_parts)

 

  • ์ถœ๋ ฅ 
๊ธ์ •์ ์ธ ๋ถ€๋ถ„:
['์ด ์ œํ’ˆ์€ ์ •๋ง ์ข‹์•˜์Šต๋‹ˆ๋‹ค.']

๋ถ€์ •์ ์ธ ๋ถ€๋ถ„:
['๋ฐฐ์†ก์ด ์กฐ๊ธˆ ๋Š๋ ธ์Šต๋‹ˆ๋‹ค.']
๋ฐ˜์‘ํ˜•

๊ณต์œ 

๋Œ“๊ธ€