django 站点地图中的动态查询集

问题描述

我有超过 100,000,000 个页面 URL，我怎样才能使 QuerySet 动态化，因为每个类都有 10,000 个唯一 URL，而无需在 10,000 个类中手动创建整数？

this.foo({a:'something',b:true,c:123});

解决方法

您应该能够将查询集拆分为多个块并生成站点地图。

作为从这个 gist 中分块查询集的示例

# utils.py

def queryset_iterator(queryset,chunk_size=1000):
    """
    Iterate over a Django Queryset ordered by the primary key
    This method loads a maximum of chunk_size (default: 1000) rows in it's
    memory at the same time while django normally would load all rows in it's
    memory. Using the iterator() method only causes it to not preload all the
    classes.
    Note that the implementation of the iterator does not support ordered query sets.
    """
        try:
            last_pk = queryset.order_by('-pk')[:1].get().pk
        except ObjectDoesNotExist:
            return

        pk = 0
        queryset = queryset.order_by('pk')
        while pk < last_pk:
            for row in queryset.filter(pk__gt=pk)[:chunk_size]:
                pk = row.pk
                yield row
            gc.collect()

您应该能够采用这种方法来生成站点地图

# sitemaps.py

from django.contrib.sitemaps import Sitemap

from appname.models import Page

from .utils import queryset_iterator


def generate_sitemaps():
    sitemaps = {}
    qs = Page.objects.all()
    i = 1
    for chunk in queryset_iterator(qs,chunk_size=10000):
        _sitemap = PageSitemap(items=chunk)
        sitemaps[f"page_{i}"] = _sitemap
        i += 1
    return sitemaps


class PageSitemap(Sitemap):
    changefreq = "never"
    priority = 0.5

    def __init__(self,items=None):
        if items:
            self.items = items
        super().__init__()

    def items(self):
        if self.items:
            return items
        else:
            return Page.objects.all()

# urls

from django.contrib.sitemaps.views import sitemap
from django.views.decorators.cache import cache_page

from .sitemaps import generate_sitemaps


urlpatterns = [
    path(
        'sitemap.xml',cache_page(timeout=60 * 60,cache='pages')(sitemap),{'sitemaps': generate_sitemaps()}
    ),]