PHP循环中的内存使用量不断增加

6
我有一个脚本需要执行,它会完成很多任务并且会重复执行大约21k次。问题在于,对于每个索引,我都要做好几件不同的事情。每个索引都是数据库中的一个产品,我从API获取数据来更新价格并保存产品等等。我有几个区域,在几乎每个方法调用之前和之后都调用了memory_get_usage(),我发现每次调用都会增加内存使用量。没有一个方法比其他方法更占用内存或者差异不明显。
我已经尝试在循环结束时取消所有变量,也尝试将它们设置为null,但无论如何,每次迭代内存限制都会不断增加。
是否有任何方法可以清除这些内存?我认为取消变量应该能够释放内存,但似乎并不是这样?
编辑: 我忘记提到我开始调查的原因是服务器上出现了内存限制错误。它并不总是在相同的时间点发生,甚至不是每次运行都会发生。这就是我试图调查的原因。
这个脚本需要大约一个小时才能运行完,我通常在早上运行它,此时没有其他任务在进行,而且现在它只在一个暂存服务器上运行,所以没有人在访问该服务器。
我可以发布代码,但它非常大。
<?php


if( !function_exists('memory_get_usage') ){
    include('function.php');
}
echo "At the start we're using (in bytes): ",
     memory_get_usage() , "\n\n";

$path = realpath(dirname(__FILE__) . '/../../../../Mage.php');
require_once($path);
Mage::app();
require_once '/lib/ProductUpdate.php';
echo "Starting product update process \n\n";
$productUpdate = new ProductUpdate();
$dealerStoreId = 3;
$volumeDiscountGroupId = 4;
$retailGroupId = Mage_Customer_Model_Group::CUST_GROUP_ALL;
$wholesaleGroupId = 2;


echo "Grabbing all products \n\n";
Mage::app()->setCurrentStore(Mage_Core_Model_App::ADMIN_STORE_ID);

// get the products from the InOrder stored procedure qty since datetime and don't pass a date to get all products, also pass the id of the cron job
$ioProducts = $productUpdate->getProductUpdateProducts('WEB2');
echo "---------------------------\n\n";
echo "Begin Updating Products \n\n";
echo "---------------------------\n\n";
$productCount = 0;
$productUpdate->saveScriptStarted(2);
echo "Before we go into the initial loop we are using (in bytes): ",
     memory_get_usage() , "\n\n";
foreach ($ioProducts as $ioProduct) {
    $updateProduct = false;
    $updateTierPrice = false;
    $sku = trim($ioProduct['inp_short_item_number']) . trim($ioProduct['isc_SIZE']) . trim($ioProduct['isc_COLOR']);
    echo "Checking item number " . $sku . " \n\n";
    echo "Before Loading Product " . $sku .  " we are using (in bytes): ",
     memory_get_usage() , "\n\n";
    $product = $productUpdate->getProduct();
    $productId = $product->getIdBySku($sku);
    echo "After Getting Id from sku " . $sku .  " we are using (in bytes): ",
     memory_get_usage() , "\n\n";
    if ($productId) {
        //$product = $productUpdate->getProduct()->load($productId);
        echo "After Loading Product " . $sku .  " we are using (in bytes): ",
            memory_get_usage() , "\n\n";
        echo "WE HAVE A PRODUCT!: " . $product->getName() . "\n\n";

        try {
            echo "Before Getting Additional Info from InOrder for Product " . $sku .  " we are using (in bytes): ",
            memory_get_usage() , "\n\n";

            // Since the product is same for parent products as it is for children you should just be able to get the price  of the parent and use that.
            $additionalInfo = $productUpdate->getItemDetails($ioProduct['inp_short_item_number'], 'WEB2');

            echo "After Getting Additional Info from InOrder for Product " . $sku .  " we are using (in bytes): ",
            memory_get_usage() , "\n\n";

            echo "Before Getting Extra Charges from InOrder for Product " . $sku .  " we are using (in bytes): ",
            memory_get_usage() , "\n\n";

            $oversizeCharge = $productUpdate->getExtraCharges($ioProduct['inp_short_item_number']);

            echo "After Getting Extra Charges from InOrder for Product " . $sku .  " we are using (in bytes): ",
            memory_get_usage() , "\n\n";
        } catch (Exception $e) {
            echo $e->getMessage() . "\n\n";
            continue;
        }

        if (is_array($additionalInfo) && count($additionalInfo) > 0) {
            if (isset($oversizeCharge[0]['Shipping Unit Charge']) && $product->getOversizeCharge() != $oversizeCharge[0]['Shipping Unit Charge']) {
                $product->setOversizeCharge($oversizeCharge[0]['Shipping Unit Charge']);
                $updateProduct = true;
                unset($oversizeCharge);
            }
            if ($product->getPrice() != $additionalInfo[0]['pri_current_price']) {
                $product->setPrice($additionalInfo[0]['pri_current_price']);
                $updateProduct = true;
                unset($additionalInfo);
            }
            echo "Before Setting Stock Status for Product " . $sku .  " we are using (in bytes): ",
            memory_get_usage() , "\n\n";

            $product = $productUpdate->setStockStatus($product, $ioProduct);

            echo "After Setting Stock Status for Product " . $sku .  " we are using (in bytes): ",
            memory_get_usage() , "\n\n";

            if ($product->getNeedsUpdate()) {
                $updateProduct = true;
            }

            if ($updateProduct) {
                try{
                    echo "Before Saving Product " . $sku .  " we are using (in bytes): ",
                    memory_get_usage() , "\n\n";

                   $productUpdate->saveProduct($product, $ioProduct);

                    echo "After Saving Product " . $sku .  " we are using (in bytes): ",
                    memory_get_usage() , "\n\n";
                }catch (Exception $e){
                    echo $e->getMessage() . "\n\n";
                    continue;
                }
            }


            // Go through  and do the same thing for the other 2 web classes to set pricing for the Dealer and Volume wholesale customers
            $updateProduct = false;
            try {
                echo "Before getting Tier Price info for " . $sku .  " we are using (in bytes): ",
                memory_get_usage() , "\n\n";

                $product = $productUpdate->getProduct()->setStoreId($dealerStoreId)->load($productId);
                $additionalInfo = $productUpdate->getItemDetails($ioProduct['inp_short_item_number'], 'WEB3');
                //$additionalTierInfo = $productUpdate->getItemDetails($ioProduct['inp_short_item_number'], 'WEB4');

                // Get Real Tier Prices based on Customer Type
                $retailPriceBreaks = $productUpdate->getItemPriceBreaks($ioProduct['inp_short_item_number'], Mage::getStoreConfig(Lancaster_InOrder_Helper_Data::XML_PATH_RETAIL_PRICE_LIST));
                $wholesalePriceBreaks = $productUpdate->getItemPriceBreaks($ioProduct['inp_short_item_number'], Mage::getStoreConfig(Lancaster_InOrder_Helper_Data::XML_PATH_WHOLESALE_PRICE_LIST));
                $volumeWholesalePriceBreaks = $productUpdate->getItemPriceBreaks($ioProduct['inp_short_item_number'], Mage::getStoreConfig(Lancaster_InOrder_Helper_Data::XML_PATH_VOLUME_WHOLESALE_PRICE_LIST));

                echo "After getting Tier Price infor for " . $sku .  " we are using (in bytes): ",
                memory_get_usage() , "\n\n";
            } catch (Exception $e) {
                echo $e->getMessage() . "\n\n";
                continue;
            }


            if ($product->getPrice() != $additionalInfo[0]['pri_current_price']) {
                $product->setPrice($additionalInfo[0]['pri_current_price']);
                $updateProduct = true;
            }

            //The only way to setup multiple price for one website is to set a tier price so we set it to a specific group and the dealer site then go through and set all the other real tier prices
            $tierPriceInfo = $product->getData('tier_price');
            if (!empty($tierPriceInfo)) {
                echo "Before looping through Tier Price infor for " . $sku .  " we are using (in bytes): ",
                memory_get_usage() , "\n\n";
                foreach ($tierPriceInfo as $tierPrice) {
                    if ($tierPrice["website_id"] == $dealerStoreId &&
                        $tierPrice["cust_group"] == $volumeDiscountGroupId &&
                        $tierPrice["price_qty"] == '1' &&
                        $tierPrice["price"] != $additionalTierInfo[0]['pri_current_price']) {
                        $updateTierPrice = true;
                    }
                    //todo need to do some refinement to the following, was rushed to put out the logic need to fix so it doesn't update everytime
                    // need to find if any of the tier prices do not match price as well if there is a price break in InOrder but not in Magento

                    if (!$updateTierPrice ) {

                        $updateRetail = isUpdateTierPrices($retailPriceBreaks, $tierPrice, $retailGroupId);
                        $updateWholesale = isUpdateTierPrices($wholesalePriceBreaks, $tierPrice, $wholesaleGroupId);
                        $updateVolWholesale = isUpdateTierPrices($volumeWholesalePriceBreaks, $tierPrice, $volumeDiscountGroupId);
                        if (
                            (count($retailPriceBreaks) > 0 && !$updateRetail['priceTierExists']) &&
                            (count($wholesalePriceBreaks) > 0 && !$updateWholesale['priceTierExists']) &&
                            (count($volumeWholesalePriceBreaks) > 0 && !$updateVolWholesale['priceTierExists'])) {
                             $updateTierPrice = true;
                        }

                        if(($updateRetail['updateTierPrice'] || $updateWholesale['updateTierPrice'] || $updateVolWholesale['updateTierPrice'])){
                            $updateTierPrice = true;
                        }
                    }
                }
                unset($tierPriceInfo);
                echo "After looping through Tier Price infor for " . $sku .  " we are using (in bytes): ",
                memory_get_usage() , "\n\n";
            }
            else {
                $updateTierPrice = true;
            }
            if ($updateTierPrice) {
                echo "Before setting whether we update Tier Price for " . $sku .  " we are using (in bytes): ",
                memory_get_usage() , "\n\n";
                //construct the tier price
                $website_id = Mage::getModel('core/store')->load($dealerStoreId)->getWebsiteId();
                $tierPrices = array(array(
                                        'website_id' => $website_id,
                                        'cust_group' => $volumeDiscountGroupId,
                                        'price_qty' => '1',
                                        'price' => $additionalTierInfo[0]['pri_current_price']
                                    ));

                updateTierPrices($retailPriceBreaks, $retailGroupId, $tierPrices);
                updateTierPrices($wholesalePriceBreaks, $wholesaleGroupId, $tierPrices);
                updateTierPrices($volumeWholesalePriceBreaks, $volumeDiscountGroupId, $tierPrices);

                $product->setData('tier_price', $tierPrices);
                $updateProduct = true;
                unset($website_id);
                echo "After setting whether we update Tier Price for " . $sku .  " we are using (in bytes): ",
                memory_get_usage() , "\n\n";
            }

            if ($updateProduct) {
                try{
                     echo "Before saving product for Tiered Pricing for " . $sku .  " we are using (in bytes): ",
                    memory_get_usage() , "\n\n";
                  // $productUpdate->saveProduct($product, $ioProduct);
                    echo "After saving product for Tiered Pricing for " . $sku .  " we are using (in bytes): ",
                    memory_get_usage() , "\n\n";
                }catch (Exception $e){
                    echo $e->getMessage() . "\n\n";
                    continue;
                }

            }
        }
    }
    $retailPriceBreaks = null;
    $wholesalePriceBreaks = null;
    $volumeWholesalePriceBreaks = null;
    $oversizeCharge = null;
    $additionalTierInfo = null;
    $additionalInfo = null;
    $product = null;
    $productCount++;
    echo $productCount . " Products have been proceessed \n\n";
}
echo "After running through all products we are using (in bytes): ",
                    memory_get_usage() , "\n\n";
echo "Peak memory usage for product update scrip (in bytes): ",
                    memory_get_peak_usage() , "\n\n";

除非您发布您的代码,否则很难帮助您。 - Cfreak
已更新我的代码,不过很大。 - dan.codes
2个回答

5

在PHP中增加内存使用是正常的。取消设置变量并不会立即释放它占用的内存,它只是将其标记为可重新使用。在某个时刻,PHP将决定运行垃圾收集器,这时内存才真正被释放。

除非您实际上遇到“内存不足”致命错误,否则不必担心。PHP尽力防止OOM发生,但它不会每次取消设置变量时都进行非常昂贵的垃圾收集运行。如果那样做,性能会彻底停滞。


还要注意的是,当脚本/线程完成时应释放内存,否则PHP的垃圾回收机制会处理它。 - Brian
3
抱歉,我忘记提到并将在我的帖子中更新,我在整个脚本中遇到了内存不足的问题。它并不总是发生,并且不总是在同一位置发生。因此,我开始放置获取使用情况的调用以查看它来自哪里,并注意到它正在各处增加。 - dan.codes
你用的是哪个版本的php?5.3版有一个改进的垃圾回收器,可以解决循环引用问题,这在之前的版本中导致了许多内存泄漏。 - Marc B
2
你可以尝试使用 gc_collect_cycles()(http://php.net/manual/en/function.gc-collect-cycles.php)来强制运行GC,但这只适用于5.3版本。 - Marc B
我能提出的建议就是将更新过程分成更小的块。不要一次性处理所有21k个产品,而是分批处理1000个,并在每个循环之间重新实例化控制对象。Mage::系统中可能存在内存泄漏问题,完全杀死/重启对象可能会让RAM使用量真正降低。 - Marc B
显示剩余2条评论

0

不看你的代码,我猜测 PHP 的垃圾回收(释放未使用内存)没有在脚本运行的时间内运行。

总之,这种行为是可以接受和预期的。只要你没有遇到内存溢出错误,就应该没问题。


网页内容由stack overflow 提供, 点击上面的
可以查看英文原文,
原文链接