借鉴GeeCache实现了MyCache(https://geektutu.com/post/geecache.html)

主要功能:

1.实现了fifo和lru两种淘汰算法
2.基于一致性哈希支持分布式并发读写
3.使用了singleflight防止缓存击穿

结构:

consistenthash/consistenthash.go实现了一致性哈希,对外暴露的数据结构及方法包括:

// Map constains all hashed keys
type Map struct {
	hash     Hash           // Hash function
	replicas int            // Virtual node multiple
	keys     []int          // Sorted
	hashMap  map[int]string // Mapping Table of Virtual Node and Real Node
}

// New creates a Map instance
func New(replicas int, fn Hash) *Map 

// Add adds some keys to the hash.
func (m *Map) Add(keys ...string)

// Get gets the closest item in the hash to the provided key.
func (m *Map) Get(key string) string
eliminationstrategy/eliminationstrategy.go实现了fifo和lru淘汰算法的单机非并发cache,对外暴露的数据结构及方法包括:

// Cache is a LRU/FIFO cache. It is not safe for concurrent access.
type Cache struct {
	eliminationstrategy int
	maxBytes            int64
	nbytes              int64
	ll                  *list.List
	cache               map[string]*list.Element
	// optional and executed when an entry is purged.
	OnEvicted func(key string, value Value)
}

// New is the Constructor of Cache
func New(maxBytes int64, eliminationstrategy int, onEvicted func(string, Value)) *Cache

// Get look ups a key's value
func (c *Cache) Get(key string) (value Value, ok bool) 

// Add adds a value to the cache.
func (c *Cache) Add(key string, value Value)
singleflight/singleflight.go实现了只向远端节点发起一次请求以防止缓存击穿,对外暴露的数据结构及方法包括:

type Group struct {
	mu sync.Mutex // protects m
	m  map[string]*call
}

// Do only initiate one request to the same key
func (g *Group) Do(key string, fn func() (interface{}, error)) (interface{}, error)
byteview.go实现了 ByteView 用来表示缓存值,对外暴露的数据结构及方法包括:

// A ByteView holds an immutable view of bytes.
type ByteView struct {
	b []byte
}

// Len returns the view's length
func (v ByteView) Len() int

// ByteSlice returns a copy of the data as a byte slice.
func (v ByteView) ByteSlice() []byte

// String returns the data as a string, making a copy if necessary.
func (v ByteView) String() string 
cache.go为eliminationstrategy/eliminationstrategy.go中实现的cache添加了并发能力,对外暴露的数据结构及方法包括:

type cache struct {
	mu         sync.Mutex
	es         int
	esCache    *eliminationstrategy.Cache
	cacheBytes int64
}

// Add() includes New()
func (c *cache) Add(key string, value ByteView)

func (c *cache) Get(key string) (value ByteView, ok bool)
http.go提供了mycache的分布式http请求,peers.go提供了PeerPicker和PeerGetter接口

mycache.go组成了最终的分布式缓存,对外暴露的数据结构及方法包括:

// A Getter loads data for a key.
type Getter interface {
	Get(key string) ([]byte, error)
}

// A GetterFunc implements Getter with a function.
type GetterFunc func(key string) ([]byte, error)

// Get implements Getter interface function
func (f GetterFunc) Get(key string) ([]byte, error) {
return f(key)
}

// A Group is a cache namespace and associated data loaded spread over
type Group struct {
    name      string
    getter    Getter
    mainCache cache
    peers     PeerPicker
    // use singleflight.Group to make sure that each key is only fetched once
    loader *singleflight.Group
}

// NewGroup create a new instance of Group
func NewGroup(name string, es int, cacheBytes int64, getter Getter) *Group

// GetGroup returns the named group previously created with NewGroup, or
// nil if there's no such group.
func GetGroup(name string) *Group

// Get value for a key from cache
func (g *Group) Get(key string) (ByteView, error)

// RegisterPeers registers a PeerPicker for choosing remote peer
func (g *Group) RegisterPeers(peers PeerPicker)

GitHub

View Github