MySQL 创建临时表 + 事务会导致死锁。

16

我有一个复杂的查询,创建了一个临时表,可能需要5秒或更长时间才能运行。这似乎导致在同时运行类似表上的另一个事务时发生死锁。我无法在本地重现,但在生产环境中,我每隔几天就会遇到一次。(我记录了mysql错误)

查询比较复杂(在页面底部显示),但您不需要理解其逻辑; 只需知道它从许多表格和连接中进行选择,并且可能需要一段时间才能运行。

我还有一个事务,插入了许多相同的表格。偶尔会出现mysql错误1213:尝试获取锁定时发现死锁;请尝试重新启动事务。

以下是事务的伪代码:

START TRANSACTION
INSERT INTO phppos_sales
INSERT MANY RECORDS INTO phppos_sales_items
INSERT MANY RECORDS INTO phppos_sales_items_taxes
INSERT MANY RECORDS INTO phppos_sales_payments
END TRANSACTION

如何解决这个死锁问题?我尝试将隔离级别更改为READ UNCOMMITTED,但mysql设置不允许这样做;而且我需要在许多环境中使其工作,在那里我无法控制服务器。

更改隔离级别时出现错误:

无法执行语句:由于BINLOG_FORMAT = STATEMENT且至少有一个表使用仅适用于基于行的记录的存储引擎,因此无法写入二进制日志。当事务隔离级别为READ COMMITTED或READ UNCOMMITTED时,InnoDB仅限于行记录。

INNODB引擎状态:

mysql> SHOW ENGINE INNODB STATUS;
| Type   | Name | Status                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
| InnoDB |      | 
=====================================
140520 12:00:17 INNODB MONITOR OUTPUT
=====================================
Per second averages calculated from the last 15 seconds
-----------------
BACKGROUND THREAD
-----------------
srv_master_thread loops: 1766819 1_second, 1766816 sleeps, 167043 10_second, 100947 background, 100945 flush
srv_master_thread log flush and writes: 1776023
----------
SEMAPHORES
----------
OS WAIT ARRAY INFO: reservation count 236559, signal count 288374
Mutex spin waits 546890, rounds 1796579, OS waits 33216
RW-shared spins 205374, rounds 5519210, OS waits 176937
RW-excl spins 5661, rounds 841678, OS waits 23933
Spin rounds per wait: 3.29 mutex, 26.87 RW-shared, 148.68 RW-excl
------------------------
LATEST FOREIGN KEY ERROR
------------------------
140520 11:27:44 Transaction:
TRANSACTION 86D125F, ACTIVE 0 sec inserting
mysql tables in use 1, locked 1
15 lock struct(s), heap size 3112, 6 row lock(s), undo log entries 2
MySQL thread id 1910245, OS thread handle 0x7fbf0042e700, query id 56114114 php-pos-web 10.181.16.33 phppoint update
INSERT INTO `phppos_sales_items_taxes` (`sale_id`, `item_id`, `line`, `name`, `percent`, `cumulative`) VALUES (11763, 1115, 3, 'PST', '8.000', '0')
Foreign key constraint fails for table `phppoint_fatpanda`.`phppos_sales_items_taxes`:
,
  CONSTRAINT `phppos_sales_items_taxes_ibfk_1` FOREIGN KEY (`sale_id`) REFERENCES `phppos_sales_items` (`sale_id`)
Trying to add in child table, in index `PRIMARY` tuple:
DATA TUPLE: 8 fields;
 0: len 4; hex 80002df3; asc   - ;;
 1: len 4; hex 8000045b; asc    [;;
 2: len 4; hex 80000003; asc     ;;
 3: len 3; hex 505354; asc PST;;
 4: len 8; hex 8000000000080000; asc         ;;
 5: len 6; hex 0000086d125f; asc    m _;;
 6: len 7; hex 00000000000000; asc        ;;
 7: len 4; hex 80000000; asc     ;;

But in parent table `phppoint_fatpanda`.`phppos_sales_items`, in index `PRIMARY`,
the closest match we can find is record:
PHYSICAL RECORD: n_fields 11; compact format; info bits 0
 0: len 4; hex 80002df1; asc   - ;;
 1: len 4; hex 8000049a; asc     ;;
 2: len 4; hex 80000001; asc     ;;
 3: len 6; hex 0000086cfd29; asc    l );;
 4: len 7; hex f400000216012c; asc       ,;;
 5: len 0; hex ; asc ;;
 6: len 0; hex ; asc ;;
 7: len 11; hex 8000000000010000000000; asc            ;;
 8: len 11; hex 8000000000100000000000; asc            ;;
 9: len 11; hex 80000000002d0000000000; asc      -     ;;
 10: len 4; hex 80000000; asc     ;;

------------------------
LATEST DETECTED DEADLOCK
------------------------
140520 11:27:44
*** (1) TRANSACTION:
TRANSACTION 86D11A3, ACTIVE 2 sec fetching rows
mysql tables in use 9, locked 9
LOCK WAIT 364 lock struct(s), heap size 47544, 80177 row lock(s)
MySQL thread id 1910243, OS thread handle 0x7fbeb2090700, query id 56113840 10.181.26.42 phppoint Copying to tmp table

CREATE TEMPORARY TABLE phppos_sales_items_temp
        (SELECT phppos_sales.deleted as deleted,phppos_sales.deleted_by as deleted_by, sale_time, date(sale_time) as sale_date, phppos_sales_items.sale_id, comment,payment_type, customer_id, employee_id, 
        phppos_items.item_id, NULL as item_kit_id, supplier_id, quantity_purchased, item_cost_price, item_unit_price, category, 
        discount_percent, (item_unit_price*quantity_purchased-item_unit_price*quantity_purchased*discount_percent/100) as subtotal,
        phppos_sales_items.line as line, serialnumber, phppos_sales_items.description as description,
        (item_unit_price*quantity_purchased-item_unit_price*quantity_purchased*discount_percent/100)+(item_unit_price*quantity_purchased-item_unit_price*quantity_purchased*discount_percent/100)*(SUM(CASE WHEN cumulative != 1 THEN percent ELSE 0 END)/100) 
        +(((item_unit_price*quantity_purchased-item_unit_price*quanti
*** (1) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 454941 page no 114 n bits 408 index `location_id` of table `phppoint_fatpanda`.`phppos_sales` trx id 86D11A3 lock mode S waiting
Record lock, heap no 335 PHYSICAL RECORD: n_fields 2; compact format; info bits 0
 0: len 4; hex 80000001; asc     ;;
 1: len 4; hex 80002df3; asc   - ;;

*** (2) TRANSACTION:
TRANSACTION 86D125D, ACTIVE 0 sec inserting
mysql tables in use 1, locked 1
23 lock struct(s), heap size 3112, 12 row lock(s), undo log entries 10
MySQL thread id 1910245, OS thread handle 0x7fbf0042e700, query id 56114091 php-pos-web 10.181.16.33 phppoint update

INSERT INTO `phppos_sales_items_taxes` (`sale_id`, `item_id`, `line`, `name`, `percent`, `cumulative`) VALUES (11763, 1178, 2, 'GST', '5.000', '0')
*** (2) HOLDS THE LOCK(S):
RECORD LOCKS space id 454941 page no 114 n bits 408 index `location_id` of table `phppoint_fatpanda`.`phppos_sales` trx id 86D125D lock_mode X locks rec but not gap
Record lock, heap no 335 PHYSICAL RECORD: n_fields 2; compact format; info bits 0
 0: len 4; hex 80000001; asc     ;;
 1: len 4; hex 80002df3; asc   - ;;

*** (2) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 454945 page no 386 n bits 288 index `PRIMARY` of table `phppoint_fatpanda`.`phppos_sales_items_taxes` trx id 86D125D lock_mode X insert intention waiting
Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0
 0: len 8; hex 73757072656d756d; asc supremum;;

*** WE ROLL BACK TRANSACTION (2)
------------
TRANSACTIONS
------------
Trx id counter 86E47F7
Purge done for trx's n:o < 86E45C0 undo n:o < 0
History list length 1418
LIST OF TRANSACTIONS FOR EACH SESSION:
---TRANSACTION 86E47F6, not started
MySQL thread id 1913171, OS thread handle 0x7fbeb2090700, query id 56205829 10.181.26.42 phppoint
---TRANSACTION 0, not started
MySQL thread id 1913095, OS thread handle 0x7fbf005b4700, query id 56205830 localhost root
SHOW ENGINE INNODB STATUS
--------
FILE I/O
--------
I/O thread 0 state: waiting for completed aio requests (insert buffer thread)
I/O thread 1 state: waiting for completed aio requests (log thread)
I/O thread 2 state: waiting for completed aio requests (read thread)
I/O thread 3 state: waiting for completed aio requests (read thread)
I/O thread 4 state: waiting for completed aio requests (read thread)
I/O thread 5 state: waiting for completed aio requests (read thread)
I/O thread 6 state: waiting for completed aio requests (write thread)
I/O thread 7 state: waiting for completed aio requests (write thread)
I/O thread 8 state: waiting for completed aio requests (write thread)
I/O thread 9 state: waiting for completed aio requests (write thread)
Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] ,
 ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0
Pending flushes (fsync) log: 0; buffer pool: 0
3599456 OS file reads, 9300371 OS file writes, 3988632 OS fsyncs
0.27 reads/s, 16384 avg bytes/read, 13.07 writes/s, 7.27 fsyncs/s
-------------------------------------
INSERT BUFFER AND ADAPTIVE HASH INDEX
-------------------------------------
Ibuf: size 1, free list len 84, seg size 86, 55915 merges
merged operations:
 insert 68506, delete mark 4761, delete 38
discarded operations:
 insert 0, delete mark 0, delete 0
Hash table size 2212699, node heap has 751 buffer(s)
5050.86 hash searches/s, 624.09 non-hash searches/s
---
LOG
---
Log sequence number 184365806376
Log flushed up to   184365806376
Last checkpoint at  184365791715
0 pending log writes, 0 pending chkp writes
2149282 log i/o's done, 3.47 log i/o's/second
----------------------
BUFFER POOL AND MEMORY
----------------------
Total memory allocated 1098907648; in additional pool allocated 0
Dictionary memory allocated 62951505
Buffer pool size   65536
Free buffers       1
Database pages     64784
Old database pages 23894
Modified db pages  88
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 4073451, not young 0
0.07 youngs/s, 0.00 non-youngs/s
Pages read 3592230, created 1542046, written 6130789
0.27 reads/s, 18.00 creates/s, 6.40 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 64784, unzip_LRU len: 0
I/O sum[382]:cur[0], unzip sum[0]:cur[0]
--------------
ROW OPERATIONS
--------------
0 queries inside InnoDB, 0 queries in queue
1 read views open inside InnoDB
Main thread process no. 15866, id 140457065543424, state: sleeping
Number of rows inserted 77431960, updated 1673031, deleted 160450, read 4825684197
1103.93 inserts/s, 2.53 updates/s, 0.00 deletes/s, 7772.15 reads/s
----------------------------
END OF INNODB MONITOR OUTPUT
============================
 |
+-------

查询语句是死锁的一部分:(与上述事务一起)

CREATE temporary TABLE phppos_sales_items_temp 
  (SELECT 
  phppos_sales.deleted                                              AS deleted, 
          phppos_sales.deleted_by                                   AS deleted_by, 
          sale_time, 
          Date(sale_time)                                           AS sale_date, 
          phppos_sales_items.sale_id, 
          comment, 
          payment_type, 
          customer_id, 
          employee_id, 
          phppos_items.item_id, 
          NULL                                                       AS item_kit_id, 
          supplier_id, 
          quantity_purchased, 
          item_cost_price, 
          item_unit_price, 
          category, 
          discount_percent, 
          ( item_unit_price * quantity_purchased - 
            item_unit_price * quantity_purchased * 
            discount_percent / 100 ) AS subtotal, 
          phppos_sales_items.line                                     AS line, 
          serialnumber, 
          phppos_sales_items.description                              AS description, 
          ( item_unit_price * quantity_purchased - 
            item_unit_price * quantity_purchased * 
            discount_percent / 100 ) + ( 
          item_unit_price * quantity_purchased - 
          item_unit_price * quantity_purchased * 
          discount_percent 
          / 100 ) * ( Sum(CASE 
                            WHEN cumulative != 1 THEN percent 
                            ELSE 0 
                          end) / 100 ) + ( ( ( 
          item_unit_price * quantity_purchased 
          - 
          item_unit_price * quantity_purchased 
          * 
          discount_percent / 100 ) * ( 
                                           Sum(CASE 
                                                 WHEN cumulative != 1 THEN 
                                                 percent 
                                                 ELSE 0 
                                               end) / 100 ) + ( 
                                           item_unit_price * quantity_purchased 
                                           - 
                                                            item_unit_price * 
                                                            quantity_purchased 
                                                            * 
                                                            discount_percent / 
                                                            100 
                                                              ) 
                                     ) 
                                           * ( Sum(CASE 
                                                                            WHEN 
                                                   cumulative = 1 THEN percent 
                                                                            ELSE 
                                                   0 
                                                                            end) 
                                             ) 
                                           / 100 ) 
                                                                       AS total, 
          ( item_unit_price * quantity_purchased - 
            item_unit_price * quantity_purchased * 
            discount_percent / 100 ) * ( 
          Sum(CASE 
                WHEN cumulative != 1 THEN percent 
                ELSE 0 
              end) / 100 ) + ( ( ( item_unit_price * quantity_purchased - 
                                   item_unit_price * quantity_purchased * 
                                   discount_percent / 100 ) * ( Sum( 
                                                        CASE 
                                                          WHEN cumulative != 1 
                                                        THEN 
                                                          percent 
                                                          ELSE 0 
                                                        end) / 100 ) + ( 
                                 item_unit_price * quantity_purchased 
                                 - 
                                 item_unit_price * quantity_purchased 
                                 * 
                                 discount_percent / 100 ) ) * ( Sum( 
                                 CASE 
                                 WHEN cumulative = 1 THEN percent 
                                 ELSE 0 
                                 end) ) / 100 )          AS tax, 
          ( item_unit_price * quantity_purchased - 
            item_unit_price * quantity_purchased * 
            discount_percent / 100 ) - ( 
          item_cost_price * quantity_purchased )                      AS profit 
   FROM   phppos_sales_items 
          INNER JOIN phppos_sales 
                  ON phppos_sales_items.sale_id = phppos_sales.sale_id 
          INNER JOIN phppos_items 
                  ON phppos_sales_items.item_id = phppos_items.item_id 
          LEFT OUTER JOIN phppos_suppliers 
                       ON phppos_items.supplier_id = phppos_suppliers.person_id 
          LEFT OUTER JOIN phppos_sales_items_taxes 
                       ON phppos_sales_items.sale_id = 
                          phppos_sales_items_taxes.sale_id 
                          AND phppos_sales_items.item_id = 
                              phppos_sales_items_taxes.item_id 
                          AND phppos_sales_items.line = 
                              phppos_sales_items_taxes.line 
   WHERE  sale_time BETWEEN "2014-04-01 00:00:00" AND "2014-04-30 23:59:59" 
          AND phppos_sales.location_id = '1' 
          AND phppos_sales.store_account_payment = 0 
   GROUP  BY sale_id, 
             item_id, 
             line) 
  UNION ALL 
  (SELECT phppos_sales.deleted 
          AS 
          deleted 
          , 
          phppos_sales.deleted_by 
          AS deleted_by, 
          sale_time, 
          Date(sale_time) 
          AS 
          sale_date, 
          phppos_sales_item_kits.sale_id, 
          comment, 
          payment_type, 
          customer_id, 
          employee_id, 
          NULL 
          AS 
          item_id, 
          phppos_item_kits.item_kit_id, 
          '' 
          AS 
          supplier_id, 
          quantity_purchased, 
          item_kit_cost_price, 
          item_kit_unit_price, 
          category, 
          discount_percent, 
          ( item_kit_unit_price * quantity_purchased - 
            item_kit_unit_price * quantity_purchased * discount_percent / 100 ) 
          AS 
          subtotal, 
          phppos_sales_item_kits.line 
          AS 
          line, 
          '' 
          AS 
          serialnumber, 
          phppos_sales_item_kits.description 
          AS 
          description, 
          ( item_kit_unit_price * quantity_purchased - 
            item_kit_unit_price * quantity_purchased * discount_percent / 100 ) 
          + 
          ( item_kit_unit_price * quantity_purchased - 
            item_kit_unit_price * quantity_purchased * discount_percent / 100 ) *
       ( Sum(  CASE   WHEN  cumulative != 1 THEN percent    ELSE 0  end) / 100 ) 
+ ( ( (  item_kit_unit_price * quantity_purchased  - 
item_kit_unit_price * quantity_purchased * discount_percent / 100 ) *
( Sum(CASE 
WHEN cumulative != 1 THEN percent 
ELSE 0 
      end) 
  / 
  100 ) + ( item_kit_unit_price * quantity_purchased - 
                      item_kit_unit_price * quantity_purchased * 
                      discount_percent / 100 ) ) * ( 
Sum( 
                               CASE 
                                 WHEN cumulative = 1 THEN percent 
                                 ELSE 0 
                               end) ) / 100 )                         AS total, 
( item_kit_unit_price * quantity_purchased - 
  item_kit_unit_price * quantity_purchased * discount_percent / 100 ) * ( Sum( 
CASE 
WHEN cumulative != 1 THEN percent 
ELSE 0 
end) / 100 ) + ( ( ( item_kit_unit_price * quantity_purchased - 
                                        item_kit_unit_price * quantity_purchased 
                                        * 
                                        discount_percent 
                                        / 100 ) * ( Sum( 
                   CASE 
                     WHEN cumulative != 1 THEN percent 
                     ELSE 0 
                   end) / 100 ) + ( 
                                      item_kit_unit_price * quantity_purchased 
                                      - 
                                                 item_kit_unit_price * 
                                                 quantity_purchased 
                                                 * discount_percent / 100 ) ) * 
                 ( 
                                  Sum(CASE 
                                        WHEN cumulative = 1 THEN percent 
                                        ELSE 0 
                                      end) ) / 100 )                  AS tax, 
( item_kit_unit_price * quantity_purchased - 
  item_kit_unit_price * quantity_purchased * discount_percent / 100 ) - ( 
item_kit_cost_price * quantity_purchased )                            AS profit 
 FROM   phppos_sales_item_kits 
        INNER JOIN phppos_sales 
                ON phppos_sales_item_kits.sale_id = phppos_sales.sale_id 
        INNER JOIN phppos_item_kits 
                ON phppos_sales_item_kits.item_kit_id = 
                   phppos_item_kits.item_kit_id 
        LEFT OUTER JOIN phppos_sales_item_kits_taxes 
                     ON phppos_sales_item_kits.sale_id = 
                        phppos_sales_item_kits_taxes.sale_id 
                        AND phppos_sales_item_kits.item_kit_id = 
                            phppos_sales_item_kits_taxes.item_kit_id 
                        AND phppos_sales_item_kits.line = 
                            phppos_sales_item_kits_taxes.line 
 WHERE  sale_time BETWEEN "2014-04-01 00:00:00" AND "2014-04-30 23:59:59" 
        AND phppos_sales.location_id = '1' 
        AND phppos_sales.store_account_payment = 0 
 GROUP  BY sale_id, 
           item_kit_id, 
           line) 
ORDER  BY sale_id, 
          line; 

1
你真的在 SQL 中计算吗?当你有 PHP 可以使用时,就用 PHP 进行计算,只使用 SQL 进行 CRUD 操作。 - Xatenev
2
这与问题无关。我在询问死锁问题。 - Chris Muench
4
这就是为什么我把它写成了评论,而不是答案 :) - Xatenev
你当前的隔离级别是什么?这对回答问题有很大影响。 - Brendan F
默认的可重复读取(REPEATABLE READ) - Chris Muench
1个回答

28

根本原因

当你将SELECT语句与写入语句(如INSERT INTO...CREATE TABLE AS...)结合使用时,MySQL必须对涉及到的表进行共享锁定

你有另一个并发事务(2),它持有一个对表phppos_sales排他锁,因此事务(1)无法获得其S锁,而事务(1)等待。

然后事务(2)请求对表phppos_sales_items_taxes进行X锁定。但是事务(1)已经在队列中等待获取该表的S锁,事务(2)必须在队列后面等待。

因此事务(2)正在等待事务(1),而事务(1)正在等待事务(2)。这是一个典型的死锁。

这只会每隔几天发生一次,因为它取决于事务(2)在事务(1)开始其SELECT之前获取其第一个锁定在phppos_sales上。然后,在事务(1)将其S锁请求排队之后,事务(2)尝试获取其在phppos_sales_items_taxes上的第二个锁定。

换句话说,这是一种竞态条件,而这些条件很难复现。

解决方法

如果事务(2)能够作为一个原子操作请求获取其需要的所有表的锁,则事务(1)就没有机会在锁请求之间插入。

你可以通过显式使用LOCK TABLES来实现此目的:

START TRANSACTION
LOCK TABLES phppos_sales WRITE, phppos_sales_items WRITE, 
    phppos_sales_items_taxes WRITE, ...other table(s)...
INSERT INTO phppos_sales
INSERT MANY RECORDS INTO phppos_sales_items
INSERT MANY RECORDS INTO phppos_sales_items_taxes
INSERT MANY RECORDS INTO phppos_sales_payments
UNLOCK TABLES;
COMMIT;

这意味着执行长时间的SELECT的事务(1)必须等待执行INSERT并释放表锁的事务(2)完成。如果先进行SELECT,那么事务(2)就必须等待其完成。

解决方法

如果避免使用CREATE TABLE…SELECTINSERT INTO... SELECT,则可以在没有锁竞争的情况下填充临时表。也就是说,将SELECT的结果集获取到应用程序中,然后将这些行INSERT到临时表中。这样,SELECT就不需要任何S锁。

您还可以在存储过程中使用游标执行相同的操作。

另一种解决方法

如@BrendanF所述,您还可以将事务隔离级别更改为READ-COMMITTED,而不是默认的REPEATABLE-READ。您可以在全局范围内更改默认的事务隔离级别,也可以逐个会话地更改隔离级别。这会对事务的语义产生一些变化,因此您应该阅读有关差异的文档。

但这可以消除插入/选择操作期间从表中读取时SELECT需要进行S锁的需要。


在第一种解决方案中,当我解锁表时,我需要指定要解锁哪些表吗?当我在插入事务上有锁时,其他事务会发生什么?它会等待还是超时? - Chris Muench
我已经将更改实施到生产环境中。我会再等几天,以确保死锁错误不会再次出现。我还不得不添加读取锁。总共我不得不对27个表进行读写锁定! - Chris Muench
INSERT INTO ... SELECT 在所有隔离级别中是否需要 S-Lock,还是只在 read-commited 以上的隔离级别中需要?我无法想象为什么在 read-commited 或 read-uncommited 中需要。 - Brendan F
@BrendanF,不是所有的事务隔离级别都会在源行上创建S锁,只有可重复读和串行化隔离级别才会创建。但可重复读是默认的事务隔离级别。 - Bill Karwin
1
@LorenzMeyer,在进行INSERT INTO...SELECTCREATE TABLE...SELECT操作的事务开始之前,您应该设置READ-COMMITTED - Bill Karwin
显示剩余6条评论

网页内容由stack overflow 提供, 点击上面的
可以查看英文原文,
原文链接