尽管查看了其他线程,但我仍然有点困惑,以下两种情况是否成立:
- 所有异步代码都是多线程的
- 所有多线程函数都是异步的
我的初步猜测是两者都不是,并且适当的异步代码应该能够在一个线程中运行-但是可以通过添加线程来改进,例如:
因此,我构建了这个玩具示例:
from threading import *
from queue import Queue
import time
def do_something_with_io_lag(in_work):
out = in_work
# Imagine we do some work that involves sending
# something over the internet and processing the output
# once it arrives
time.sleep(0.5) # simulate IO lag
print("Hello, bee number: ",
str(current_thread().name).replace("Thread-",""))
class WorkerBee(Thread):
def __init__(self, q):
Thread.__init__(self)
self.q = q
def run(self):
while True:
# Get some work from the queue
work_todo = self.q.get()
# This function will simiulate I/O lag
do_something_with_io_lag(work_todo)
# Remove task from the queue
self.q.task_done()
if __name__ == '__main__':
def time_me(nmbr):
number_of_worker_bees = nmbr
worktodo = ['some input for work'] * 50
# Create a queue
q = Queue()
# Fill with work
[q.put(onework) for onework in worktodo]
# Launch processes
for _ in range(number_of_worker_bees):
t = WorkerBee(q)
t.start()
# Block until queue is empty
q.join()
# Run this code in serial mode (just one worker)
%time time_me(nmbr=1)
# Wall time: 25 s
# Basically 50 requests * 0.5 seconds IO lag
# For me everything gets processed by bee number: 59
# Run this code using multi-tasking (launch 50 workers)
%time time_me(nmbr=50)
# Wall time: 507 ms
# Basically the 0.5 second IO lag + 0.07 seconds it took to launch them
# Now everything gets processed by different bees
它是异步的吗?
对我来说,这段代码似乎不是异步的,因为它是我的示例图中的第3个图。I/O调用会阻塞线程(尽管我们没有感觉到,因为它们在并行阻塞)。
然而,如果是这种情况,我很困惑为什么requests-futures被认为是异步的,因为它只是ThreadPoolExecutor的包装器:
with concurrent.futures.ThreadPoolExecutor(max_workers=20) as executor:
future_to_url = {executor.submit(load_url, url, 10): url for url in get_urls()}
for future in concurrent.futures.as_completed(future_to_url):
url = future_to_url[future]
try:
data = future.result()
这个功能能在单线程上运行吗?
特别是与asyncio相比,这意味着它可以单线程运行。
有两种方法可以让单处理器上的程序“同时做多件事情”。多线程编程是最简单和最流行的方法,但还有另一种非常不同的技术,可以让你几乎获得多线程的所有优势,而不实际使用多个线程。如果您的程序主要是I/O绑定,则这确实是唯一实用的方法。如果您的程序是处理器绑定的,则抢占式调度线程可能是您真正需要的。然而,网络服务器很少是处理器绑定的。