Django Tastypie Postgres 服务器响应缓慢

Question

Django Tastypie Postgres 服务器响应缓慢

3

问题：

服务器响应调用（由opbeat，DHC chrome客户端测量）大约需要500毫秒到5000毫秒。Postgres查询（每个调用的总数）比它们对应的响应时间快20倍至50倍。为什么这么慢？？？

背景：

我在webfaction上运行一个实时的tastypie API（共享实例，1GB RAM :( ）为我们的移动应用程序。 Django 1.8，Python 2.7，0.12.2.dev0 tastypie，PostgreSQL 9.4.，CentOS7 数据库有大约40个表，~2G，6k用户（可能有1/3是“活跃”的），数据库位于单独的共享webfaction框中。虽然我们在api中有一些过程性代码，但许多缓慢的调用只是针对用户资源的GET请求-最大/最慢的是大约50kb的记录，JSON响应中的~350个对象。此调用的postgres时间为〜20-50ms，在快速专用开发服务器上的最快时间为2秒-2gig RAM，2个处理器-并且在现场为〜4秒，最慢的时间为10-16秒在两个盒子上。运行在Apache / mod_wsgi上，HTTPS，（apache的基准测试很快），没有gzip。 httpd.conf设置似乎很好。

简而言之：

99％的时间在tastypie.resources.wrapper中 - 我没有改变这个代码。这些调用只是返回资源的调用。数据库查询很快 Web服务器似乎很快。

诊断：

Django调试工具栏（用于开发专用框上最慢操作的最快调用）

用户CPU时间1926.979毫秒系统CPU时间27.074毫秒总CPU时间1954.053毫秒经过的时间1980.884毫秒上下文切换71个自愿，44个非自愿

httpd.conf

LoadModule authz_core_module modules/mod_authz_core.so

LoadModule dir_module        modules/mod_dir.so

LoadModule env_module        modules/mod_env.so

LoadModule log_config_module modules/mod_log_config.so

LoadModule mime_module       modules/mod_mime.so

LoadModule rewrite_module    modules/mod_rewrite.so

LoadModule setenvif_module   modules/mod_setenvif.so

LoadModule wsgi_module       modules/mod_wsgi.so

LoadModule unixd_module      modules/mod_unixd.so

...

KeepAlive Off

SetEnvIf X-Forwarded-SSL on HTTPS=1

ServerLimit 1

StartServers 1

MaxRequestWorkers 5

MinSpareThreads 1

MaxSpareThreads 3

ThreadsPerChild 5

...

WSGIApplicationGroup %{GLOBAL}

WSGIDaemonProcess djangoapp processes=2 threads=8 python-path=...

WSGIProcessGroup djangoapp

WSGIRestrictEmbedded On

WSGILazyInitialization On

Opbeat在实时服务器上的表现：（这个抓取有更多/更长的数据库查询，因为我正在测试删除select_related()/prefetch_related() - 它们有助于db查询时间，但并没有使总时间减少太多）：

什么鬼，要发帖子需要10点声望？

最后的想法：

tastypie是真的慢吗？当然不是。虽然更好的盒子运行得更快，但Python用2秒钟输出了数据库20ms的内容，这是否现实呢？
是的，当我把它放在一个更快、专用的开发盒子上时，时间缩短了一些，但它们仍然是SQL调用所需时间的10-50倍，例如，对于一个资源GET，最快的用时也要2秒，而数据库只需要<50毫秒。所以，这似乎不仅仅是一个资源问题，据我所知。
我试图链接到Amazon RDS PostgreSQL数据库，但它的响应时间慢了20倍（微型/免费层）且整体往返时间也慢了。
感谢您的帮助和关注-

- WimTar

1个回答

网页内容由stack overflow 提供, 点击上面的

可以查看英文原文，
原文链接

- Tomasz Jakub Rup · Answer 1

在我的调查中，Tastypie很慢，因为函数django.core.urlresolvers.reverse很慢。

我放弃了resource_uri并以这种方式修补了Tastypie：

django_app/monkey/tastypie.py

from __future__ import absolute_import

from django.core.exceptions import ObjectDoesNotExist
from tastypie.bundle import Bundle
from tastypie.exceptions import ApiFieldError


def dehydrate(self, bundle, for_list=True):
    if not bundle.obj or not bundle.obj.pk:
        if not self.null:
            raise ApiFieldError(
                "The model '%r' does not have a primary key and can not be used in a ToMany context." % bundle.obj)

        return []

    the_m2ms = None
    previous_obj = bundle.obj
    attr = self.attribute

    if isinstance(self.attribute, basestring):
        attrs = self.attribute.split('__')
        the_m2ms = bundle.obj

        for attr in attrs:
            previous_obj = the_m2ms
            try:
                the_m2ms = getattr(the_m2ms, attr, None)
            except ObjectDoesNotExist:
                the_m2ms = None

            if not the_m2ms:
                break

    elif callable(self.attribute):
        the_m2ms = self.attribute(bundle)

    if not the_m2ms:
        if not self.null:
            raise ApiFieldError(
                "The model '%r' has an empty attribute '%s' and doesn't allow a null value." % (previous_obj, attr))

        return []

    self.m2m_resources = []
    m2m_dehydrated = []

    # TODO: Also model-specific and leaky. Relies on there being a
    # ``Manager`` there.
    m2ms = the_m2ms.all() if for_list else the_m2ms.get_query_set().all()
    for m2m in m2ms:
        m2m_resource = self.get_related_resource(m2m)
        m2m_bundle = Bundle(obj=m2m, request=bundle.request)
        self.m2m_resources.append(m2m_resource)
        m2m_dehydrated.append(self.dehydrate_related(m2m_bundle, m2m_resource, for_list=for_list))

    return m2m_dehydrated


def _build_reverse_url(self, name, args=None, kwargs=None):
    return kwargs.get('pk')


def get_via_uri(self, uri, request=None):
    bundle = self.build_bundle(request=request)
    return self.obj_get(bundle=bundle, pk=uri)


def build_related_resource(self, value, request=None, related_obj=None, related_name=None):
    """
    Returns a bundle of data built by the related resource, usually via
    ``hydrate`` with the data provided.

    Accepts either a URI, a data dictionary (or dictionary-like structure)
    or an object with a ``pk``.
    """
    self.fk_resource = self.to_class()
    kwargs = {
        'request': request,
        'related_obj': related_obj,
        'related_name': related_name,
    }

    if isinstance(value, Bundle):
        # Already hydrated, probably nested bundles. Just return.
        return value
    elif isinstance(value, (basestring, int)):
        # We got a URI. Load the object and assign it.
        return self.resource_from_uri(self.fk_resource, value, **kwargs)
    elif isinstance(value, Bundle):
        # We got a valid bundle object, the RelatedField had full=True
        return value
    elif hasattr(value, 'items'):
        # We've got a data dictionary.
        # Since this leads to creation, this is the only one of these
        # methods that might care about "parent" data.
        return self.resource_from_data(self.fk_resource, value, **kwargs)
    elif hasattr(value, 'pk'):
        # We've got an object with a primary key.
        return self.resource_from_pk(self.fk_resource, value, **kwargs)
    else:
        raise ApiFieldError("The '%s' field was given data that was not a URI, not a dictionary-alike and does not have a 'pk' attribute: %s." % (self.instance_name, value))

django_app/monkey/init.py

def patch_tastypie():
    from tastypie.fields import ToManyField, RelatedField
    from tastypie.resources import Resource, ResourceOptions
    from monkey.tastypie import dehydrate, _build_reverse_url, get_via_uri, build_related_resource

    setattr(ToManyField, 'dehydrate', dehydrate)
    setattr(Resource, '_build_reverse_url', _build_reverse_url)
    setattr(Resource, 'get_via_uri', get_via_uri)
    setattr(ResourceOptions, 'include_resource_uri', False)
    setattr(RelatedField, 'build_related_resource', build_related_resource)

django_app/init.py

from __future__ import absolute_import

from .monkey import patch_tastypie


patch_tastypie()

虽不完美，但可以加速 Tastypie 10-20%