<em>Mac</em>Book项目 2009年学校开始实施<em>Mac</em>Book项目,所有师生配备一本<em>Mac</em>Book,并同步更新了校园无线网络。学校每周进行电脑技术更新,每月发送技术支持资料,极大改变了教学及学习方式。因此2011
2021-06-01 09:32:01
前段時間碰到個奇怪的索引失效的問題,實際情況類似下面這樣:
bill=# begin; BEGIN bill=*# create index idx_t1 on t1(id); CREATE INDEX bill=*# explain select * from t1 where id = 1; QUERY PLAN ---------------------------------------------------- Seq Scan on t1 (cost=0.00..25.88 rows=6 width=36) Filter: (id = 1) (2 rows) bill=*# end; COMMIT bill=# explain select * from t1 where id = 1; QUERY PLAN --------------------------------------------------------------------- Bitmap Heap Scan on t1 (cost=1.50..7.01 rows=6 width=36) Recheck Cond: (id = 1) -> Bitmap Index Scan on idx_t1 (cost=0.00..1.50 rows=6 width=0) Index Cond: (id = 1) (4 rows)
很顯然的問題就是,我在事務中建立了索引,卻沒辦法使用。但是當事務提交了後便可以正常使用了,這是什麼情況呢?
這個其實和pg_index中indcheckxmin屬性有關,關於這個欄位的解釋如下:
If true, queries must not use the index until the xmin of this pg_index row is below their TransactionXmin event horizon, because the table may contain broken HOT chains with incompatible rows that they can see
經檢查也確實如此:
bill=*# select indcheckxmin from pg_index where indexrelid = 'idx_t1'::regclass; indcheckxmin -------------- t (1 row)
那麼問題來了,什麼情況下建立索引時會將索引的該屬性設定為true呢?
從前面官方檔案對於該欄位的解釋,如果表中包含broken HOT chains 則會為true,那什麼是broken HOT chains ?似乎和HOT機制有關。那是不是隻有存在broken HOT chains 才會設定為true呢?
這裡就不賣關子了,直接給出結論,然後我們再去一一驗證。
經測試發現,以下兩種情況會導致索引的indcheckxmin設定為true:
場景一:broken HOT chains
這種情況,只要在當前事務中表中存在HOT更新的行時就會存在。那麼什麼時候會進行HOT更新呢?兩個前提:
既然如此,實際中常見的兩種情況就是:
例子:
表中插入10條資料,自然只有1個page:
bill=# insert into t1 select generate_series(1,10),md5(random()::text); INSERT 0 10
進行更新:
bill=# update t1 set info = 'bill' where id = 10; UPDATE 1
檢視發現的確是HOT更新:
關於t_infomask2欄位的解釋這裡就不再贅述。
接下來我們建立索引:
可以發現indcheckxmin被設定為true,在當前事務中索引不可用。
經過驗證,在index_build階段,判斷到BrokenHotChain,便將indcheckxmin修改為true。
具體的修改程式碼如下:
/*此時indexInfo->ii_BrokenHotChain已被修改為true */ if ((indexInfo->ii_BrokenHotChain || EarlyPruningEnabled(heapRelation)) && !isreindex && !indexInfo->ii_Concurrent) { Oid indexId = RelationGetRelid(indexRelation); Relation pg_index; HeapTuple indexTuple; Form_pg_index indexForm; pg_index = table_open(IndexRelationId, RowExclusiveLock); indexTuple = SearchSysCacheCopy1(INDEXRELID, ObjectIdGetDatum(indexId)); if (!HeapTupleIsValid(indexTuple)) elog(ERROR, "cache lookup failed for index %u", indexId); indexForm = (Form_pg_index) GETSTRUCT(indexTuple); /* If it's a new index, indcheckxmin shouldn't be set ... */ Assert(!indexForm->indcheckxmin); /*將indcheckxmin修改為true */ indexForm->indcheckxmin = true; CatalogTupleUpdate(pg_index, &indexTuple->t_self, indexTuple); heap_freetuple(indexTuple); table_close(pg_index, RowExclusiveLock); }
同樣我們也可以驗證得知,的確是因為brokenhotchains導致的indcheckxmin被設定為true。
場景二:old_snapshot_threshold
先來看例子:
最簡單的場景,完全的一張空表,在事務中建立索引indcheckxmin就會被設定為true,果然索引也是不可用。
bill=# drop table t1; DROP TABLE bill=# create table t1(id int,info text); CREATE TABLE bill=# begin; BEGIN bill=*# create index idx_t1 on t1(id); CREATE INDEX bill=*# select indcheckxmin from pg_index where indexrelid = 'idx_t1'::regclass; indcheckxmin -------------- t (1 row) bill=*# explain select * from t1 where id = 1; QUERY PLAN ---------------------------------------------------- Seq Scan on t1 (cost=0.00..25.88 rows=6 width=36) Filter: (id = 1) (2 rows)
那麼為什麼old_snapshot_threshold會產生這樣的影響呢?
經過跟蹤發現,當開啟該引數時,在事務中建立索引的snapshotdata結構如下:
(SnapshotData) $6 = {
snapshot_type = SNAPSHOT_MVCC
xmin = 856
xmax = 856
xip = 0x00007fd55c804fc0
xcnt = 0
subxip = 0x00007fd55ad5d000
subxcnt = 0
suboverflowed = false
takenDuringRecovery = false
copied = false
curcid = 1
speculativeToken = 0
vistest = NULL
active_count = 0
regd_count = 0
ph_node = {
first_child = NULL
next_sibling = NULL
prev_or_parent = NULL
}
whenTaken = 691752041261069
lsn = 208079736
}
而禁用該引數呢?
(SnapshotData) $7 = {
snapshot_type = SNAPSHOT_MVCC
xmin = 828
xmax = 828
xip = 0x00007fad31704780
xcnt = 0
subxip = 0x00007fad3155d000
subxcnt = 0
suboverflowed = false
takenDuringRecovery = false
copied = false
curcid = 1
speculativeToken = 0
active_count = 0
regd_count = 0
ph_node = {
first_child = NULL
next_sibling = NULL
prev_or_parent = NULL
}
whenTaken = 0
lsn = 0
}
可以看到,區別在於不使用該引數時,建立snapshotdata不會設定whenTaken和lsn,那麼這兩個引數是幹嘛的呢?
先來看看snapshotdata的結構:
typedef struct SnapshotData { SnapshotType snapshot_type; /* type of snapshot */ /* * The remaining fields are used only for MVCC snapshots, and are normally * just zeroes in special snapshots. (But xmin and xmax are used * specially by HeapTupleSatisfiesDirty, and xmin is used specially by * HeapTupleSatisfiesNonVacuumable.) * * An MVCC snapshot can never see the effects of XIDs >= xmax. It can see * the effects of all older XIDs except those listed in the snapshot. xmin * is stored as an optimization to avoid needing to search the XID arrays * for most tuples. */ TransactionId xmin; /* all XID < xmin are visible to me */ TransactionId xmax; /* all XID >= xmax are invisible to me */ /* * For normal MVCC snapshot this contains the all xact IDs that are in * progress, unless the snapshot was taken during recovery in which case * it's empty. For historic MVCC snapshots, the meaning is inverted, i.e. * it contains *committed* transactions between xmin and xmax. * * note: all ids in xip[] satisfy xmin <= xip[i] < xmax */ TransactionId *xip; uint32 xcnt; /* # of xact ids in xip[] */ /* * For non-historic MVCC snapshots, this contains subxact IDs that are in * progress (and other transactions that are in progress if taken during * recovery). For historic snapshot it contains *all* xids assigned to the * replayed transaction, including the toplevel xid. * * note: all ids in subxip[] are >= xmin, but we don't bother filtering * out any that are >= xmax */ TransactionId *subxip; int32 subxcnt; /* # of xact ids in subxip[] */ bool suboverflowed; /* has the subxip array overflowed? */ bool takenDuringRecovery; /* recovery-shaped snapshot? */ bool copied; /* false if it's a static snapshot */ CommandId curcid; /* in my xact, CID < curcid are visible */ /* * An extra return value for HeapTupleSatisfiesDirty, not used in MVCC * snapshots. */ uint32 speculativeToken; /* * For SNAPSHOT_NON_VACUUMABLE (and hopefully more in the future) this is * used to determine whether row could be vacuumed. */ struct GlobalVisState *vistest; /* * Book-keeping information, used by the snapshot manager */ uint32 active_count; /* refcount on ActiveSnapshot stack */ uint32 regd_count; /* refcount on RegisteredSnapshots */ pairingheap_node ph_node; /* link in the RegisteredSnapshots heap */ TimestampTz whenTaken; /* timestamp when snapshot was taken */ XLogRecPtr lsn; /* position in the WAL stream when taken */ /* * The transaction completion count at the time GetSnapshotData() built * this snapshot. Allows to avoid re-computing static snapshots when no * transactions completed since the last GetSnapshotData(). */ uint64 snapXactCompletionCount; } SnapshotData;
如上所示,TimestampTz表示snapshot何時產生的,為什麼啟用old_snapshot_threshold時會設定該值呢?
因為該值正是用來判斷快照是否過舊的:
/* * Implement slower/larger portions of TestForOldSnapshot * * Smaller/faster portions are put inline, but the entire set of logic is too * big for that. */ void TestForOldSnapshot_impl(Snapshot snapshot, Relation relation) { if (RelationAllowsEarlyPruning(relation) && (snapshot)->whenTaken < GetOldSnapshotThresholdTimestamp()) ereport(ERROR, (errcode(ERRCODE_SNAPSHOT_TOO_OLD), errmsg("snapshot too old"))); }
這樣我們也比較好理解為什麼設定了該引數時建立的索引在當前事務中不可用:
因為我們不設定該引數時,在事務中建立索引是可以保證MVCC的一致性,那麼索引便是安全可用的。
而使用引數時,由於TimestampTz被設定,資料庫會對其進行判斷該行資料是否已經過期,如果過期了那便會被清理掉,這樣對於索引來說便是不安全的,沒法保證資料的一致性,對於不是hot-safe的索引,自然要將其indcheckxmin設定為true,防止在事務中建立索引後資料實際已經過期被刪除的情況。
/* * At this moment we are sure that there are no transactions with the * table open for write that don't have this new index in their list of * indexes. We have waited out all the existing transactions and any new * transaction will have the new index in its list, but the index is still * marked as "not-ready-for-inserts". The index is consulted while * deciding HOT-safety though. This arrangement ensures that no new HOT * chains can be created where the new tuple and the old tuple in the * chain have different index keys. * * We now take a new snapshot, and build the index using all tuples that * are visible in this snapshot. We can be sure that any HOT updates to * these tuples will be compatible with the index, since any updates made * by transactions that didn't know about the index are now committed or * rolled back. Thus, each visible tuple is either the end of its * HOT-chain or the extension of the chain is HOT-safe for this index. */
總結
當pg_index的indcheckxmin欄位被設定為true時,直到此pg_index行的xmin低於查詢的TransactionXmin視界之前,查詢都不能使用此索引。
而產生這種現象主要有兩種情況:
1. 表上在當前事務中存在broken HOT chains;
2. old_snapshot_threshold被設定時。
到此這篇關於PostgreSQL索引失效會發生什麼的文章就介紹到這了,更多相關PostgreSQL索引失效內容請搜尋it145.com以前的文章或繼續瀏覽下面的相關文章希望大家以後多多支援it145.com!
相關文章
<em>Mac</em>Book项目 2009年学校开始实施<em>Mac</em>Book项目,所有师生配备一本<em>Mac</em>Book,并同步更新了校园无线网络。学校每周进行电脑技术更新,每月发送技术支持资料,极大改变了教学及学习方式。因此2011
2021-06-01 09:32:01
综合看Anker超能充系列的性价比很高,并且与不仅和iPhone12/苹果<em>Mac</em>Book很配,而且适合多设备充电需求的日常使用或差旅场景,不管是安卓还是Switch同样也能用得上它,希望这次分享能给准备购入充电器的小伙伴们有所
2021-06-01 09:31:42
除了L4WUDU与吴亦凡已经多次共事,成为了明面上的厂牌成员,吴亦凡还曾带领20XXCLUB全队参加2020年的一场音乐节,这也是20XXCLUB首次全员合照,王嗣尧Turbo、陈彦希Regi、<em>Mac</em> Ova Seas、林渝植等人全部出场。然而让
2021-06-01 09:31:34
目前应用IPFS的机构:1 谷歌<em>浏览器</em>支持IPFS分布式协议 2 万维网 (历史档案博物馆)数据库 3 火狐<em>浏览器</em>支持 IPFS分布式协议 4 EOS 等数字货币数据存储 5 美国国会图书馆,历史资料永久保存在 IPFS 6 加
2021-06-01 09:31:24
开拓者的车机是兼容苹果和<em>安卓</em>,虽然我不怎么用,但确实兼顾了我家人的很多需求:副驾的门板还配有解锁开关,有的时候老婆开车,下车的时候偶尔会忘记解锁,我在副驾驶可以自己开门:第二排设计很好,不仅配置了一个很大的
2021-06-01 09:30:48
不仅是<em>安卓</em>手机,苹果手机的降价力度也是前所未有了,iPhone12也“跳水价”了,发布价是6799元,如今已经跌至5308元,降价幅度超过1400元,最新定价确认了。iPhone12是苹果首款5G手机,同时也是全球首款5nm芯片的智能机,它
2021-06-01 09:30:45