golib is a golang utility library containing common tool functions and components summarized from personal project development experience.
Components:
- biz Business components
- codegen Code generation tools
- gconc Concurrent task pool component
- configkv Configuration management component
- dbaccess Database client components (supports MySQL, Redis, Elasticsearch)
- distlock Distributed lock component (non-reentrant)
- excel Excel read/write component
- gast AST syntax tree tool
- gauth Authentication component (includes jwtauth)
- gcrypto Encryption/decryption component
- gerror Error handling component
- glog Logging component
- gtrace OpenTelemetry Trace initialization component
- gtree Tree structure construction tool
- gutil Common utility functions collection
- task Task scheduling components
- protocol Protocol components (includes ghttp, gresty)
- ratelimit Rate limiting component
- storage Unified object storage component (supports S3, MinIO, OSS, COS, TOS)
go get github.com/morehao/golibSome tests require real database connections (MySQL, PostgreSQL, Redis, Elasticsearch). To avoid hardcoding credentials in test code, copy .env.example to .env and configure your local connection settings:
cp .env.example .env
# edit .env with your local credentials
# Run all tests (auto-loads .env via env_test.go)
go test ./...
# Run sub-package tests (inject env manually)
source .env && go test ./codegen/...The .env file is gitignored and will not be committed. If no environment variables are set, tests fall back to default values (127.0.0.1 with password 123456), so CI pipelines work without any extra setup.
For VS Code, configure .vscode/settings.json to inject environment variables:
{
"go.testEnvVars": {
"MYSQL_DSN": "root:123456@tcp(127.0.0.1:3306)/demo?charset=utf8mb4&parseTime=True",
"REDIS_ADDR": "127.0.0.1:6379",
"REDIS_PASSWORD": "123456"
}
}Test-specific environment variables:
| Variable | Description | Default |
|---|---|---|
MYSQL_DSN |
MySQL DSN for codegen tests | root:123456@tcp(127.0.0.1:3306)/demo?... |
POSTGRES_DSN |
PostgreSQL DSN for codegen tests | host=127.0.0.1 user=postgres password=123456... |
REDIS_ADDR |
Redis address | 127.0.0.1:6379 |
REDIS_PASSWORD |
Redis password | 123456 |
ELASTICSEARCH_ADDR |
Elasticsearch address | http://localhost:9200 |
biz is a business component package providing commonly used infrastructure components for business development.
- gcontext: Context utilities, including request ID, user ID, tenant ID and other context key-value definitions and formatting
- gobject: Common business objects, including user authentication info (UserClaims), operator info (OperatorBaseInfo), pagination query (PageQuery)
- gconstant: Business constant definitions, including error codes (100000 series), API versions, etc.
- gserver: Gin server related.
RouterGroupsis the unique top-level route group factory (path/v1/{app}), auto-mounting otelgin & access-log middleware; business modules register routes viaRegister(group, ...). Routing style spec: docs/router-style.md - gmiddleware: Gin middleware, including JWT authentication, CORS, access logging, Token blacklist
- gormplugin: GORM plugins, including multi-tenant plugin (automatically adds tenant_id filter conditions)
- genericdao: Generic DAO,封装基础的增删改查操作
- testkit: Testing toolkit, supporting test initializer and context building
- Business scenario-oriented, ready to use
- Unified error code specification
- Integrated JWT authentication and multi-tenant support
codegen is a code generation tool that reads database table structures and supports generating basic CRUD code, including router, controller, service, dto, model, errorCode, etc. The generated router layer registers RESTful CRUD routes (see docs/router-style.md).
- Supports MySQL database
- Supports PostgreSQL database
- Supports template customization and template parameter customization
- Supports code generation based on templates
For usage examples, refer to codegen unit tests
gconc is a unified concurrent task pool built on fixed workers and a buffered task queue.
Provides a single *Pool type with:
- non-blocking / timeout / blocking task submission
- graceful & immediate shutdown
- error collection & callback
- panic-safe worker pool
- runtime stats
- Flexible concurrency control
- Task queue management
- Graceful shutdown and error collection
- Panic-safe worker pool
- Thread-safe
For usage examples, refer to gconc usage
configkv is a configuration management component based on database key-value storage, supporting multiple data types and encryption.
- Supports json/toml/yaml/string/int/bool/float types
- Supports encrypted storage
- Based on GORM
dbaccess is a database client component collection providing encapsulation and connection management for multiple databases.
- dbgorm: MySQL/PostgreSQL database client, based on GORM
- dbredis: Redis client, based on go-redis
- dbes: Elasticsearch client, based on official client
- Unified configuration interface
- Integrated logging
- Connection pool configuration support
- Timeout control support
For usage examples, refer to dbaccess usage
distlock is a distributed lock component based on Redis, using redsync algorithm, supporting automatic renewal.
- Redis-based distributed lock (single or multi-node quorum)
- Automatic renewal (lock keepalive) with jitter, lock-loss notification via
Lost() - Non-reentrant
// 1. Create a lock factory (one per process; pass multiple clients for multi-node quorum)
factory := distlock.NewRedisStorage(redisClient)
// 2. Create a lock instance per key/TTL
lock, err := distlock.NewDistLock(factory, &distlock.Config{
Key: "order:pay:10086",
TTL: 30 * time.Second,
AutoRenewal: true,
})
if err != nil {
// invalid config: empty Key or TTL <= 0
}
// 3. Acquire (non-blocking) -> critical section -> release
if ok, err := lock.Lock(ctx); err != nil {
// storage failure
} else if !ok {
// lock not acquired (normal contention, retry later)
} else {
defer lock.Unlock(context.Background())
// critical section...
// when auto-renewal fails (lock lost), Lost() is closed — abort ASAP:
// <-lock.Lost()
}excel is a simple wrapper around excelize, supporting convenient Excel file read/write through structs.
Both reading and writing Excel require defining a struct, with struct fields specifying Excel-related information through tags (ex).
- Define Excel column mapping through struct tags
- Support reading and writing Excel files
- Support data validation based on validator
For usage examples, refer to excel usage
gast is a Go AST syntax tree operation tool, supporting AST analysis and code generation.
- Support function/method lookup
- Support interface method addition
- Support constant addition
- Syntax tree traversal and manipulation
gauth is an authentication component containing JWT authentication capabilities.
- jwtauth: Generic JWT signing and parsing, supports HS256 algorithm, supports renewal
- Generic JWT signing and parsing
- Token renewal support
- Token blacklist support
For usage examples, refer to jwtauth usage
gcrypto is an encryption/decryption component providing common symmetric and asymmetric encryption functions.
- aes: Supports AES-128/192/256, GCM mode (recommended) and CBC mode
- rsa: Supports encryption, decryption, signing, verification, PEM format keys
- bcrypt: Password hashing and verification
- Environment variable configuration for keys
- GCM mode provides authenticated encryption
- RSA supports multiple padding modes
For usage examples, refer to gcrypto usage
gerror is an error handling component providing business error code encapsulation, supporting error chains and call stacks.
- Supports errors.Is/As
- Error chain wrapping
- Call stack recording
- Business error code specification
glog is a logging component based on zap providing high-performance logging functionality.
- Console/File output support
- OTel integration
- Structured logging support
- High-performance log writing
gtrace is an OpenTelemetry Trace initialization component supporting distributed tracing.
- OTLP gRPC/HTTP export support
- Exporter disable mechanism
- Integrated zap logging
For usage examples, refer to gtrace usage
gtree is a tree structure construction tool, a generic tree data structure building library supporting building trees from node lists.
- Provides TreeNode interface, only need to implement GetKey(), GetParentKey(), IsRoot() methods
- Orphan node handling (ignore, promote to root, error)
- Circular reference detection
- Node sorting (ID, Name, Order or multi-level combination)
- Pre-order traversal and level-order traversal
gutil is a collection of common utility functions providing commonly used tool functions during development.
- Random number generation
- String processing
- Date/time operations
- Type conversion
- Slice/Map operations
- File processing
task is a task scheduling component package containing cron and async task sub-packages, both persisting execution records via GORM and integrating glog logging and gtrace distributed tracing.
- gcron: Cron tasks, based on
robfig/cron/v3, supporting second-level cron, multi-instance distributed lock mutual exclusion, and execution record persistence - gasync: Async tasks, based on
hibiken/asynq, supporting retry, timeout, delay, priority queues, execution record persistence, and cross-process trace propagation
- Second-level cron expressions and custom timezone
- Multi-instance distributed lock mutual exclusion (with optional auto-renewal)
- Per-task execution timeout and in-process overlap prevention (gcron)
- Retry, timeout, retention, and multi-queue priority
- Automatic execution record persistence (idempotent re-registration on restart)
- Automatic TraceID/RequestID/RunID injection and logging (string primary key as task/run identifier)
- Cross-process trace propagation
- Graceful shutdown:
Client.Close,Server.ShutdownContext(gasync),Scheduler.Stop(ctx)(gcron)
For usage examples, refer to task usage
protocol is a protocol-related component collection providing HTTP client encapsulation.
- ghttp: Enhanced HTTP client, supports struct mapping, connection pool, smart retry and other features
- gresty: HTTP client wrapper based on Resty, supports SSE (Server-Sent Events)
- Struct automatic mapping support
- Connection pool optimization
- Smart retry mechanism (no retry for 4xx, retry for 5xx)
- SSE long connection support
- Rich configuration options
For usage examples, refer to ghttp usage
storage is a unified object storage component supporting multiple cloud providers with a consistent API.
- AWS S3
- MinIO
- Alibaba Cloud OSS
- Tencent Cloud COS
- Volcano Engine TOS
- Unified API across all providers
- Multipart upload support
- Presigned URL generation (GET/PUT)
- Object listing with paginator
- Batch operations (delete, copy)
- URI helper for standardized resource identifiers
- Key builder with prefix, date layout, and random suffix
For usage examples, refer to storage usage
ratelimit is a rate limiting component: distributed rate limiting on Redis (redis_rate GCRA token bucket), with automatic degradation to a local token bucket when Redis is unavailable.
- Redis rate limiting (go-redis-rate, GCRA token bucket)
- Automatic fail-over to in-process rate limiting (fail-open) when Redis is down, with exponential-backoff probing and automatic switch-back on recovery
- Degradation/recovery events can be logged via
WithLogger Close()releases background goroutines
- During fail-over, each process uses its own local limiter; in multi-instance deployments the aggregate limit is roughly
instances × configured rate, and quotas are not shared across instances Rate/Burst/Period/CleanupIntervalmust be positive, otherwiseNewLimiterreturns an error