我没有使用LangChain;只是使用内置的requests模块。然而,对于生产项目,请考虑使用Flask-SSE。
def open_ai_stream(prompt):
url = config["open_ai_url"]
session = requests.Session()
payload = json.dumps({
"model": "gpt-3.5-turbo",
"messages": [
{
"role": "user",
"content": prompt
}
],
"temperature": 0,
"stream": True
})
headers = {
'Content-Type': 'application/json',
'Authorization': 'Bearer '+ config['open_ai_key']
}
with session.post(url, headers=headers, data=payload) as resp:
for line in resp.iter_lines():
if line:
yield f'data: %s\n\n' % line.decode('utf-8')
这是 Flask 路由:
@app.route("/llm-resp", methods=["POST"])
def stream_response():
prompt = request.get_json(force = True)["prompt"]
return Response(stream_with_context(open_ai_stream(query_text)),
mimetype="text/event-stream")
请查看测试结果(如果这是一个正常响应,那它在Postman上看起来会非常不同):
{{link1:
![Postman快照](https://istack.dev59.com/FlAua.webp)
}}
编辑:这里有一种稍微更好的方法,但需要添加openai库的依赖。
def chat_gpt_helper(prompt):
"""
This function returns the response from OpenAI's Gpt3.5 turbo model using the completions API
"""
try:
resp = ''
openai.api_key = os.getenv('OPEN_API_KEY')
for chunk in openai.ChatCompletion.create(
model="gpt-3.5-turbo",
messages=[{
"role": "user",
"content":prompt
}],
stream=True,
):
content = chunk["choices"][0].get("delta", {}).get("content")
if content is not None:
print(content, end='')
resp += content
yield f'data: %s\n\n' % resp
except Exception as e:
print(e)
return str(e)
其余部分保持不变。
在此处检查完整代码here:
编辑2:
如果有人无法在Postman中以块的形式查看响应,请确保您的Postman已经更新,因为在撰写本文时,这似乎是一个新功能。相关新闻发布