⚠

The web version only has simple instructions since chapter 04, while the full book has detailed explanations and background info.

0702: Store Metadata

Directory Layout

Since the DB uses multiple files, it should manage them itself. The user only specifies a directory:

type KVOptions struct {
    Dirpath string
}

type KV struct {
    Options KVOptions
    meta    KVMetaStore
    log     Log
    main    SortedFile
    // ...
}

SSTable file names are generated dynamically and stored in KVMetaData. While Log and KVMetaData use fixed file names:

    kv.log.FileName = path.Join(kv.Options.Dirpath, "kv_log")
    kv.meta.slots[0].FileName = path.Join(kv.Options.Dirpath, "meta0")
    kv.meta.slots[1].FileName = path.Join(kv.Options.Dirpath, "meta1")

Record SSTable File Names

KV.Compact() used to replace files via rename(). Now it uses KVMetaStore to record the new file name, and deletes the old file after success.

type KV struct {
    version uint64
    // ...
}

func (kv *KV) Compact() error {
    kv.version++
    sstable := fmt.Sprintf("sstable_%d", kv.version)
    file := SortedFile{FileName: path.Join(kv.Options.Dirpath, sstable)}
    // ...
}

The file name is generated by an monotonic number, which is the version in KVMetaData.

func (kv *KV) Open() error {
    // ...
    kv.version = kv.meta.Get().Version
    // ...
}

ⓘ

CodeCrafters.io has similar courses in many programming languages, including build your own Redis, SQLite, Docker, etc. It’s worth checking out.