admin/kubesphere
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

chore(deps): bump github.com/containerd/containerd from 1.6.18 to 1.7.13 (#6102)

Browse Source 
Signed-off-by: hongzhouzi <hongzhouzi@kubesphere.io>
This commit is contained in:
hongzhouzi
2024-05-09 17:31:31 +08:00
committed by GitHub
parent 80cb9a6302
commit b5015ec7b9
243 changed files with 34968 additions and 3511 deletions
Download Patch File Download Diff File Expand all files Collapse all files

0
vendor/go.opentelemetry.io/otel/exporters/otlp/internal/retry/LICENSE → vendor/github.com/AdaLogics/go-fuzz-headers/LICENSE generated vendored
View File

93
vendor/github.com/AdaLogics/go-fuzz-headers/README.md generated vendored Normal file
View File

@@ -0,0 +1,93 @@
# go-fuzz-headers
This repository contains various helper functions for go fuzzing. It is mostly used in combination with [go-fuzz](https://github.com/dvyukov/go-fuzz), but compatibility with fuzzing in the standard library will also be supported. Any coverage guided fuzzing engine that provides an array or slice of bytes can be used with go-fuzz-headers.
## Usage
Using go-fuzz-headers is easy. First create a new consumer with the bytes provided by the fuzzing engine:
```go
import (
fuzz "github.com/AdaLogics/go-fuzz-headers"
)
data := []byte{'R', 'a', 'n', 'd', 'o', 'm'}
f := fuzz.NewConsumer(data)
```
This creates a `Consumer` that consumes the bytes of the input as it uses them to fuzz different types.
After that, `f` can be used to easily create fuzzed instances of different types. Below are some examples:
### Structs
One of the most useful features of go-fuzz-headers is its ability to fill structs with the data provided by the fuzzing engine. This is done with a single line:
```go
type Person struct {
Name string
Age int
}
p := Person{}
// Fill p with values based on the data provided by the fuzzing engine:
err := f.GenerateStruct(&p)
```
This includes nested structs too. In this example, the fuzz Consumer will also insert values in `p.BestFriend`:
```go
type PersonI struct {
Name string
Age int
BestFriend PersonII
}
type PersonII struct {
Name string
Age int
}
p := PersonI{}
err := f.GenerateStruct(&p)
```
If the consumer should insert values for unexported fields as well as exported, this can be enabled with:
```go
f.AllowUnexportedFields()
```
...and disabled with:
```go
f.DisallowUnexportedFields()
```
### Other types:
Other useful APIs:
```go
createdString, err := f.GetString() // Gets a string
createdInt, err := f.GetInt() // Gets an integer
createdByte, err := f.GetByte() // Gets a byte
createdBytes, err := f.GetBytes() // Gets a byte slice
createdBool, err := f.GetBool() // Gets a boolean
err := f.FuzzMap(target_map) // Fills a map
createdTarBytes, err := f.TarBytes() // Gets bytes of a valid tar archive
err := f.CreateFiles(inThisDir) // Fills inThisDir with files
createdString, err := f.GetStringFrom("anyCharInThisString", ofThisLength) // Gets a string that consists of chars from "anyCharInThisString" and has the exact length "ofThisLength"
```
Most APIs are added as they are needed.
## Projects that use go-fuzz-headers
- [runC](https://github.com/opencontainers/runc)
- [Istio](https://github.com/istio/istio)
- [Vitess](https://github.com/vitessio/vitess)
- [Containerd](https://github.com/containerd/containerd)
Feel free to add your own project to the list, if you use go-fuzz-headers to fuzz it.
## Status
The project is under development and will be updated regularly.
## References
go-fuzz-headers' approach to fuzzing structs is strongly inspired by [gofuzz](https://github.com/google/gofuzz).

914
vendor/github.com/AdaLogics/go-fuzz-headers/consumer.go generated vendored Normal file
View File

@@ -0,0 +1,914 @@
// Copyright 2023 The go-fuzz-headers Authors.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package gofuzzheaders
import (
"archive/tar"
"bytes"
"encoding/binary"
"errors"
"fmt"
"io"
"math"
"os"
"path/filepath"
"reflect"
"strconv"
"strings"
"time"
"unsafe"
)
var (
MaxTotalLen uint32 = 2000000
maxDepth = 100
)
func SetMaxTotalLen(newLen uint32) {
MaxTotalLen = newLen
}
type ConsumeFuzzer struct {
data []byte
dataTotal uint32
CommandPart []byte
RestOfArray []byte
NumberOfCalls int
position uint32
fuzzUnexportedFields bool
curDepth int
Funcs map[reflect.Type]reflect.Value
}
func IsDivisibleBy(n int, divisibleby int) bool {
return (n % divisibleby) == 0
}
func NewConsumer(fuzzData []byte) *ConsumeFuzzer {
return &ConsumeFuzzer{
data: fuzzData,
dataTotal: uint32(len(fuzzData)),
Funcs: make(map[reflect.Type]reflect.Value),
curDepth: 0,
}
}
func (f *ConsumeFuzzer) Split(minCalls, maxCalls int) error {
if f.dataTotal == 0 {
return errors.New("could not split")
}
numberOfCalls := int(f.data[0])
if numberOfCalls < minCalls || numberOfCalls > maxCalls {
return errors.New("bad number of calls")
}
if int(f.dataTotal) < numberOfCalls+numberOfCalls+1 {
return errors.New("length of data does not match required parameters")
}
// Define part 2 and 3 of the data array
commandPart := f.data[1 : numberOfCalls+1]
restOfArray := f.data[numberOfCalls+1:]
// Just a small check. It is necessary
if len(commandPart) != numberOfCalls {
return errors.New("length of commandPart does not match number of calls")
}
// Check if restOfArray is divisible by numberOfCalls
if !IsDivisibleBy(len(restOfArray), numberOfCalls) {
return errors.New("length of commandPart does not match number of calls")
}
f.CommandPart = commandPart
f.RestOfArray = restOfArray
f.NumberOfCalls = numberOfCalls
return nil
}
func (f *ConsumeFuzzer) AllowUnexportedFields() {
f.fuzzUnexportedFields = true
}
func (f *ConsumeFuzzer) DisallowUnexportedFields() {
f.fuzzUnexportedFields = false
}
func (f *ConsumeFuzzer) GenerateStruct(targetStruct interface{}) error {
e := reflect.ValueOf(targetStruct).Elem()
return f.fuzzStruct(e, false)
}
func (f *ConsumeFuzzer) setCustom(v reflect.Value) error {
// First: see if we have a fuzz function for it.
doCustom, ok := f.Funcs[v.Type()]
if !ok {
return fmt.Errorf("could not find a custom function")
}
switch v.Kind() {
case reflect.Ptr:
if v.IsNil() {
if !v.CanSet() {
return fmt.Errorf("could not use a custom function")
}
v.Set(reflect.New(v.Type().Elem()))
}
case reflect.Map:
if v.IsNil() {
if !v.CanSet() {
return fmt.Errorf("could not use a custom function")
}
v.Set(reflect.MakeMap(v.Type()))
}
default:
return fmt.Errorf("could not use a custom function")
}
verr := doCustom.Call([]reflect.Value{v, reflect.ValueOf(Continue{
F: f,
})})
// check if we return an error
if verr[0].IsNil() {
return nil
}
return fmt.Errorf("could not use a custom function")
}
func (f *ConsumeFuzzer) fuzzStruct(e reflect.Value, customFunctions bool) error {
if f.curDepth >= maxDepth {
// return err or nil here?
return nil
}
f.curDepth++
defer func() { f.curDepth-- }()
// We check if we should check for custom functions
if customFunctions && e.IsValid() && e.CanAddr() {
err := f.setCustom(e.Addr())
if err != nil {
return err
}
}
switch e.Kind() {
case reflect.Struct:
for i := 0; i < e.NumField(); i++ {
var v reflect.Value
if !e.Field(i).CanSet() {
if f.fuzzUnexportedFields {
v = reflect.NewAt(e.Field(i).Type(), unsafe.Pointer(e.Field(i).UnsafeAddr())).Elem()
}
if err := f.fuzzStruct(v, customFunctions); err != nil {
return err
}
} else {
v = e.Field(i)
if err := f.fuzzStruct(v, customFunctions); err != nil {
return err
}
}
}
case reflect.String:
str, err := f.GetString()
if err != nil {
return err
}
if e.CanSet() {
e.SetString(str)
}
case reflect.Slice:
var maxElements uint32
// Byte slices should not be restricted
if e.Type().String() == "[]uint8" {
maxElements = 10000000
} else {
maxElements = 50
}
randQty, err := f.GetUint32()
if err != nil {
return err
}
numOfElements := randQty % maxElements
if (f.dataTotal - f.position) < numOfElements {
numOfElements = f.dataTotal - f.position
}
uu := reflect.MakeSlice(e.Type(), int(numOfElements), int(numOfElements))
for i := 0; i < int(numOfElements); i++ {
// If we have more than 10, then we can proceed with that.
if err := f.fuzzStruct(uu.Index(i), customFunctions); err != nil {
if i >= 10 {
if e.CanSet() {
e.Set(uu)
}
return nil
} else {
return err
}
}
}
if e.CanSet() {
e.Set(uu)
}
case reflect.Uint16:
newInt, err := f.GetUint16()
if err != nil {
return err
}
if e.CanSet() {
e.SetUint(uint64(newInt))
}
case reflect.Uint32:
newInt, err := f.GetUint32()
if err != nil {
return err
}
if e.CanSet() {
e.SetUint(uint64(newInt))
}
case reflect.Uint64:
newInt, err := f.GetInt()
if err != nil {
return err
}
if e.CanSet() {
e.SetUint(uint64(newInt))
}
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
newInt, err := f.GetInt()
if err != nil {
return err
}
if e.CanSet() {
e.SetInt(int64(newInt))
}
case reflect.Float32:
newFloat, err := f.GetFloat32()
if err != nil {
return err
}
if e.CanSet() {
e.SetFloat(float64(newFloat))
}
case reflect.Float64:
newFloat, err := f.GetFloat64()
if err != nil {
return err
}
if e.CanSet() {
e.SetFloat(float64(newFloat))
}
case reflect.Map:
if e.CanSet() {
e.Set(reflect.MakeMap(e.Type()))
const maxElements = 50
randQty, err := f.GetInt()
if err != nil {
return err
}
numOfElements := randQty % maxElements
for i := 0; i < numOfElements; i++ {
key := reflect.New(e.Type().Key()).Elem()
if err := f.fuzzStruct(key, customFunctions); err != nil {
return err
}
val := reflect.New(e.Type().Elem()).Elem()
if err = f.fuzzStruct(val, customFunctions); err != nil {
return err
}
e.SetMapIndex(key, val)
}
}
case reflect.Ptr:
if e.CanSet() {
e.Set(reflect.New(e.Type().Elem()))
if err := f.fuzzStruct(e.Elem(), customFunctions); err != nil {
return err
}
return nil
}
case reflect.Uint8:
b, err := f.GetByte()
if err != nil {
return err
}
if e.CanSet() {
e.SetUint(uint64(b))
}
}
return nil
}
func (f *ConsumeFuzzer) GetStringArray() (reflect.Value, error) {
// The max size of the array:
const max uint32 = 20
arraySize := f.position
if arraySize > max {
arraySize = max
}
stringArray := reflect.MakeSlice(reflect.SliceOf(reflect.TypeOf("string")), int(arraySize), int(arraySize))
if f.position+arraySize >= f.dataTotal {
return stringArray, errors.New("could not make string array")
}
for i := 0; i < int(arraySize); i++ {
stringSize := uint32(f.data[f.position])
if f.position+stringSize >= f.dataTotal {
return stringArray, nil
}
stringToAppend := string(f.data[f.position : f.position+stringSize])
strVal := reflect.ValueOf(stringToAppend)
stringArray = reflect.Append(stringArray, strVal)
f.position += stringSize
}
return stringArray, nil
}
func (f *ConsumeFuzzer) GetInt() (int, error) {
if f.position >= f.dataTotal {
return 0, errors.New("not enough bytes to create int")
}
returnInt := int(f.data[f.position])
f.position++
return returnInt, nil
}
func (f *ConsumeFuzzer) GetByte() (byte, error) {
if f.position >= f.dataTotal {
return 0x00, errors.New("not enough bytes to get byte")
}
returnByte := f.data[f.position]
f.position++
return returnByte, nil
}
func (f *ConsumeFuzzer) GetNBytes(numberOfBytes int) ([]byte, error) {
if f.position >= f.dataTotal {
return nil, errors.New("not enough bytes to get byte")
}
returnBytes := make([]byte, 0, numberOfBytes)
for i := 0; i < numberOfBytes; i++ {
newByte, err := f.GetByte()
if err != nil {
return nil, err
}
returnBytes = append(returnBytes, newByte)
}
return returnBytes, nil
}
func (f *ConsumeFuzzer) GetUint16() (uint16, error) {
u16, err := f.GetNBytes(2)
if err != nil {
return 0, err
}
littleEndian, err := f.GetBool()
if err != nil {
return 0, err
}
if littleEndian {
return binary.LittleEndian.Uint16(u16), nil
}
return binary.BigEndian.Uint16(u16), nil
}
func (f *ConsumeFuzzer) GetUint32() (uint32, error) {
u32, err := f.GetNBytes(4)
if err != nil {
return 0, err
}
return binary.BigEndian.Uint32(u32), nil
}
func (f *ConsumeFuzzer) GetUint64() (uint64, error) {
u64, err := f.GetNBytes(8)
if err != nil {
return 0, err
}
littleEndian, err := f.GetBool()
if err != nil {
return 0, err
}
if littleEndian {
return binary.LittleEndian.Uint64(u64), nil
}
return binary.BigEndian.Uint64(u64), nil
}
func (f *ConsumeFuzzer) GetBytes() ([]byte, error) {
var length uint32
var err error
length, err = f.GetUint32()
if err != nil {
return nil, errors.New("not enough bytes to create byte array")
}
if length == 0 {
length = 30
}
bytesLeft := f.dataTotal - f.position
if bytesLeft <= 0 {
return nil, errors.New("not enough bytes to create byte array")
}
// If the length is the same as bytes left, we will not overflow
// the remaining bytes.
if length != bytesLeft {
length = length % bytesLeft
}
byteBegin := f.position
if byteBegin+length < byteBegin {
return nil, errors.New("numbers overflow")
}
f.position = byteBegin + length
return f.data[byteBegin:f.position], nil
}
func (f *ConsumeFuzzer) GetString() (string, error) {
if f.position >= f.dataTotal {
return "nil", errors.New("not enough bytes to create string")
}
length, err := f.GetUint32()
if err != nil {
return "nil", errors.New("not enough bytes to create string")
}
if f.position > MaxTotalLen {
return "nil", errors.New("created too large a string")
}
byteBegin := f.position
if byteBegin >= f.dataTotal {
return "nil", errors.New("not enough bytes to create string")
}
if byteBegin+length > f.dataTotal {
return "nil", errors.New("not enough bytes to create string")
}
if byteBegin > byteBegin+length {
return "nil", errors.New("numbers overflow")
}
f.position = byteBegin + length
return string(f.data[byteBegin:f.position]), nil
}
func (f *ConsumeFuzzer) GetBool() (bool, error) {
if f.position >= f.dataTotal {
return false, errors.New("not enough bytes to create bool")
}
if IsDivisibleBy(int(f.data[f.position]), 2) {
f.position++
return true, nil
} else {
f.position++
return false, nil
}
}
func (f *ConsumeFuzzer) FuzzMap(m interface{}) error {
return f.GenerateStruct(m)
}
func returnTarBytes(buf []byte) ([]byte, error) {
return buf, nil
// Count files
var fileCounter int
tr := tar.NewReader(bytes.NewReader(buf))
for {
_, err := tr.Next()
if err == io.EOF {
break
}
if err != nil {
return nil, err
}
fileCounter++
}
if fileCounter >= 1 {
return buf, nil
}
return nil, fmt.Errorf("not enough files were created\n")
}
func setTarHeaderFormat(hdr *tar.Header, f *ConsumeFuzzer) error {
ind, err := f.GetInt()
if err != nil {
hdr.Format = tar.FormatGNU
//return nil
}
switch ind % 4 {
case 0:
hdr.Format = tar.FormatUnknown
case 1:
hdr.Format = tar.FormatUSTAR
case 2:
hdr.Format = tar.FormatPAX
case 3:
hdr.Format = tar.FormatGNU
}
return nil
}
func setTarHeaderTypeflag(hdr *tar.Header, f *ConsumeFuzzer) error {
ind, err := f.GetInt()
if err != nil {
return err
}
switch ind % 13 {
case 0:
hdr.Typeflag = tar.TypeReg
case 1:
hdr.Typeflag = tar.TypeLink
linkname, err := f.GetString()
if err != nil {
return err
}
hdr.Linkname = linkname
case 2:
hdr.Typeflag = tar.TypeSymlink
linkname, err := f.GetString()
if err != nil {
return err
}
hdr.Linkname = linkname
case 3:
hdr.Typeflag = tar.TypeChar
case 4:
hdr.Typeflag = tar.TypeBlock
case 5:
hdr.Typeflag = tar.TypeDir
case 6:
hdr.Typeflag = tar.TypeFifo
case 7:
hdr.Typeflag = tar.TypeCont
case 8:
hdr.Typeflag = tar.TypeXHeader
case 9:
hdr.Typeflag = tar.TypeXGlobalHeader
case 10:
hdr.Typeflag = tar.TypeGNUSparse
case 11:
hdr.Typeflag = tar.TypeGNULongName
case 12:
hdr.Typeflag = tar.TypeGNULongLink
}
return nil
}
func (f *ConsumeFuzzer) createTarFileBody() ([]byte, error) {
return f.GetBytes()
/*length, err := f.GetUint32()
if err != nil {
return nil, errors.New("not enough bytes to create byte array")
}
// A bit of optimization to attempt to create a file body
// when we don't have as many bytes left as "length"
remainingBytes := f.dataTotal - f.position
if remainingBytes <= 0 {
return nil, errors.New("created too large a string")
}
if f.position+length > MaxTotalLen {
return nil, errors.New("created too large a string")
}
byteBegin := f.position
if byteBegin >= f.dataTotal {
return nil, errors.New("not enough bytes to create byte array")
}
if length == 0 {
return nil, errors.New("zero-length is not supported")
}
if byteBegin+length >= f.dataTotal {
return nil, errors.New("not enough bytes to create byte array")
}
if byteBegin+length < byteBegin {
return nil, errors.New("numbers overflow")
}
f.position = byteBegin + length
return f.data[byteBegin:f.position], nil*/
}
// getTarFileName is similar to GetString(), but creates string based
// on the length of f.data to reduce the likelihood of overflowing
// f.data.
func (f *ConsumeFuzzer) getTarFilename() (string, error) {
return f.GetString()
/*length, err := f.GetUint32()
if err != nil {
return "nil", errors.New("not enough bytes to create string")
}
// A bit of optimization to attempt to create a file name
// when we don't have as many bytes left as "length"
remainingBytes := f.dataTotal - f.position
if remainingBytes <= 0 {
return "nil", errors.New("created too large a string")
}
if f.position > MaxTotalLen {
return "nil", errors.New("created too large a string")
}
byteBegin := f.position
if byteBegin >= f.dataTotal {
return "nil", errors.New("not enough bytes to create string")
}
if byteBegin+length > f.dataTotal {
return "nil", errors.New("not enough bytes to create string")
}
if byteBegin > byteBegin+length {
return "nil", errors.New("numbers overflow")
}
f.position = byteBegin + length
return string(f.data[byteBegin:f.position]), nil*/
}
type TarFile struct {
Hdr *tar.Header
Body []byte
}
// TarBytes returns valid bytes for a tar archive
func (f *ConsumeFuzzer) TarBytes() ([]byte, error) {
numberOfFiles, err := f.GetInt()
if err != nil {
return nil, err
}
var tarFiles []*TarFile
tarFiles = make([]*TarFile, 0)
const maxNoOfFiles = 100
for i := 0; i < numberOfFiles%maxNoOfFiles; i++ {
var filename string
var filebody []byte
var sec, nsec int
var err error
filename, err = f.getTarFilename()
if err != nil {
var sb strings.Builder
sb.WriteString("file-")
sb.WriteString(strconv.Itoa(i))
filename = sb.String()
}
filebody, err = f.createTarFileBody()
if err != nil {
var sb strings.Builder
sb.WriteString("filebody-")
sb.WriteString(strconv.Itoa(i))
filebody = []byte(sb.String())
}
sec, err = f.GetInt()
if err != nil {
sec = 1672531200 // beginning of 2023
}
nsec, err = f.GetInt()
if err != nil {
nsec = 1703980800 // end of 2023
}
hdr := &tar.Header{
Name: filename,
Size: int64(len(filebody)),
Mode: 0o600,
ModTime: time.Unix(int64(sec), int64(nsec)),
}
if err := setTarHeaderTypeflag(hdr, f); err != nil {
return []byte(""), err
}
if err := setTarHeaderFormat(hdr, f); err != nil {
return []byte(""), err
}
tf := &TarFile{
Hdr: hdr,
Body: filebody,
}
tarFiles = append(tarFiles, tf)
}
var buf bytes.Buffer
tw := tar.NewWriter(&buf)
defer tw.Close()
for _, tf := range tarFiles {
tw.WriteHeader(tf.Hdr)
tw.Write(tf.Body)
}
return buf.Bytes(), nil
}
// This is similar to TarBytes, but it returns a series of
// files instead of raw tar bytes. The advantage of this
// api is that it is cheaper in terms of cpu power to
// modify or check the files in the fuzzer with TarFiles()
// because it avoids creating a tar reader.
func (f *ConsumeFuzzer) TarFiles() ([]*TarFile, error) {
numberOfFiles, err := f.GetInt()
if err != nil {
return nil, err
}
var tarFiles []*TarFile
tarFiles = make([]*TarFile, 0)
const maxNoOfFiles = 100
for i := 0; i < numberOfFiles%maxNoOfFiles; i++ {
filename, err := f.getTarFilename()
if err != nil {
return tarFiles, err
}
filebody, err := f.createTarFileBody()
if err != nil {
return tarFiles, err
}
sec, err := f.GetInt()
if err != nil {
return tarFiles, err
}
nsec, err := f.GetInt()
if err != nil {
return tarFiles, err
}
hdr := &tar.Header{
Name: filename,
Size: int64(len(filebody)),
Mode: 0o600,
ModTime: time.Unix(int64(sec), int64(nsec)),
}
if err := setTarHeaderTypeflag(hdr, f); err != nil {
hdr.Typeflag = tar.TypeReg
}
if err := setTarHeaderFormat(hdr, f); err != nil {
return tarFiles, err // should not happend
}
tf := &TarFile{
Hdr: hdr,
Body: filebody,
}
tarFiles = append(tarFiles, tf)
}
return tarFiles, nil
}
// CreateFiles creates pseudo-random files in rootDir.
// It creates subdirs and places the files there.
// It is the callers responsibility to ensure that
// rootDir exists.
func (f *ConsumeFuzzer) CreateFiles(rootDir string) error {
numberOfFiles, err := f.GetInt()
if err != nil {
return err
}
maxNumberOfFiles := numberOfFiles % 4000 // This is completely arbitrary
if maxNumberOfFiles == 0 {
return errors.New("maxNumberOfFiles is nil")
}
var noOfCreatedFiles int
for i := 0; i < maxNumberOfFiles; i++ {
// The file to create:
fileName, err := f.GetString()
if err != nil {
if noOfCreatedFiles > 0 {
// If files have been created, we don't return an error.
break
} else {
return errors.New("could not get fileName")
}
}
if strings.Contains(fileName, "..") || (len(fileName) > 0 && fileName[0] == 47) || strings.Contains(fileName, "\\") {
continue
}
fullFilePath := filepath.Join(rootDir, fileName)
// Find the subdirectory of the file
if subDir := filepath.Dir(fileName); subDir != "" && subDir != "." {
// create the dir first; avoid going outside the root dir
if strings.Contains(subDir, "../") || (len(subDir) > 0 && subDir[0] == 47) || strings.Contains(subDir, "\\") {
continue
}
dirPath := filepath.Join(rootDir, subDir)
if _, err := os.Stat(dirPath); os.IsNotExist(err) {
err2 := os.MkdirAll(dirPath, 0o777)
if err2 != nil {
continue
}
}
fullFilePath = filepath.Join(dirPath, fileName)
} else {
// Create symlink
createSymlink, err := f.GetBool()
if err != nil {
if noOfCreatedFiles > 0 {
break
} else {
return errors.New("could not create the symlink")
}
}
if createSymlink {
symlinkTarget, err := f.GetString()
if err != nil {
return err
}
err = os.Symlink(symlinkTarget, fullFilePath)
if err != nil {
return err
}
// stop loop here, since a symlink needs no further action
noOfCreatedFiles++
continue
}
// We create a normal file
fileContents, err := f.GetBytes()
if err != nil {
if noOfCreatedFiles > 0 {
break
} else {
return errors.New("could not create the file")
}
}
err = os.WriteFile(fullFilePath, fileContents, 0o666)
if err != nil {
continue
}
noOfCreatedFiles++
}
}
return nil
}
// GetStringFrom returns a string that can only consist of characters
// included in possibleChars. It returns an error if the created string
// does not have the specified length.
func (f *ConsumeFuzzer) GetStringFrom(possibleChars string, length int) (string, error) {
if (f.dataTotal - f.position) < uint32(length) {
return "", errors.New("not enough bytes to create a string")
}
output := make([]byte, 0, length)
for i := 0; i < length; i++ {
charIndex, err := f.GetInt()
if err != nil {
return string(output), err
}
output = append(output, possibleChars[charIndex%len(possibleChars)])
}
return string(output), nil
}
func (f *ConsumeFuzzer) GetRune() ([]rune, error) {
stringToConvert, err := f.GetString()
if err != nil {
return []rune("nil"), err
}
return []rune(stringToConvert), nil
}
func (f *ConsumeFuzzer) GetFloat32() (float32, error) {
u32, err := f.GetNBytes(4)
if err != nil {
return 0, err
}
littleEndian, err := f.GetBool()
if err != nil {
return 0, err
}
if littleEndian {
u32LE := binary.LittleEndian.Uint32(u32)
return math.Float32frombits(u32LE), nil
}
u32BE := binary.BigEndian.Uint32(u32)
return math.Float32frombits(u32BE), nil
}
func (f *ConsumeFuzzer) GetFloat64() (float64, error) {
u64, err := f.GetNBytes(8)
if err != nil {
return 0, err
}
littleEndian, err := f.GetBool()
if err != nil {
return 0, err
}
if littleEndian {
u64LE := binary.LittleEndian.Uint64(u64)
return math.Float64frombits(u64LE), nil
}
u64BE := binary.BigEndian.Uint64(u64)
return math.Float64frombits(u64BE), nil
}
func (f *ConsumeFuzzer) CreateSlice(targetSlice interface{}) error {
return f.GenerateStruct(targetSlice)
}

62
vendor/github.com/AdaLogics/go-fuzz-headers/funcs.go generated vendored Normal file
View File

@@ -0,0 +1,62 @@
// Copyright 2023 The go-fuzz-headers Authors.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package gofuzzheaders
import (
"fmt"
"reflect"
)
type Continue struct {
F *ConsumeFuzzer
}
func (f *ConsumeFuzzer) AddFuncs(fuzzFuncs []interface{}) {
for i := range fuzzFuncs {
v := reflect.ValueOf(fuzzFuncs[i])
if v.Kind() != reflect.Func {
panic("Need only funcs!")
}
t := v.Type()
if t.NumIn() != 2 || t.NumOut() != 1 {
fmt.Println(t.NumIn(), t.NumOut())
panic("Need 2 in and 1 out params. In must be the type. Out must be an error")
}
argT := t.In(0)
switch argT.Kind() {
case reflect.Ptr, reflect.Map:
default:
panic("fuzzFunc must take pointer or map type")
}
if t.In(1) != reflect.TypeOf(Continue{}) {
panic("fuzzFunc's second parameter must be type Continue")
}
f.Funcs[argT] = v
}
}
func (f *ConsumeFuzzer) GenerateWithCustom(targetStruct interface{}) error {
e := reflect.ValueOf(targetStruct).Elem()
return f.fuzzStruct(e, true)
}
func (c Continue) GenerateStruct(targetStruct interface{}) error {
return c.F.GenerateStruct(targetStruct)
}
func (c Continue) GenerateStructWithCustom(targetStruct interface{}) error {
return c.F.GenerateWithCustom(targetStruct)
}

556
vendor/github.com/AdaLogics/go-fuzz-headers/sql.go generated vendored Normal file
View File

@@ -0,0 +1,556 @@
// Copyright 2023 The go-fuzz-headers Authors.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package gofuzzheaders
import (
"fmt"
"strings"
)
// returns a keyword by index
func getKeyword(f *ConsumeFuzzer) (string, error) {
index, err := f.GetInt()
if err != nil {
return keywords[0], err
}
for i, k := range keywords {
if i == index {
return k, nil
}
}
return keywords[0], fmt.Errorf("could not get a kw")
}
// Simple utility function to check if a string
// slice contains a string.
func containsString(s []string, e string) bool {
for _, a := range s {
if a == e {
return true
}
}
return false
}
// These keywords are used specifically for fuzzing Vitess
var keywords = []string{
"accessible", "action", "add", "after", "against", "algorithm",
"all", "alter", "always", "analyze", "and", "as", "asc", "asensitive",
"auto_increment", "avg_row_length", "before", "begin", "between",
"bigint", "binary", "_binary", "_utf8mb4", "_utf8", "_latin1", "bit",
"blob", "bool", "boolean", "both", "by", "call", "cancel", "cascade",
"cascaded", "case", "cast", "channel", "change", "char", "character",
"charset", "check", "checksum", "coalesce", "code", "collate", "collation",
"column", "columns", "comment", "committed", "commit", "compact", "complete",
"compressed", "compression", "condition", "connection", "constraint", "continue",
"convert", "copy", "cume_dist", "substr", "substring", "create", "cross",
"csv", "current_date", "current_time", "current_timestamp", "current_user",
"cursor", "data", "database", "databases", "day", "day_hour", "day_microsecond",
"day_minute", "day_second", "date", "datetime", "dec", "decimal", "declare",
"default", "definer", "delay_key_write", "delayed", "delete", "dense_rank",
"desc", "describe", "deterministic", "directory", "disable", "discard",
"disk", "distinct", "distinctrow", "div", "double", "do", "drop", "dumpfile",
"duplicate", "dynamic", "each", "else", "elseif", "empty", "enable",
"enclosed", "encryption", "end", "enforced", "engine", "engines", "enum",
"error", "escape", "escaped", "event", "exchange", "exclusive", "exists",
"exit", "explain", "expansion", "export", "extended", "extract", "false",
"fetch", "fields", "first", "first_value", "fixed", "float", "float4",
"float8", "flush", "for", "force", "foreign", "format", "from", "full",
"fulltext", "function", "general", "generated", "geometry", "geometrycollection",
"get", "global", "gtid_executed", "grant", "group", "grouping", "groups",
"group_concat", "having", "header", "high_priority", "hosts", "hour", "hour_microsecond",
"hour_minute", "hour_second", "if", "ignore", "import", "in", "index", "indexes",
"infile", "inout", "inner", "inplace", "insensitive", "insert", "insert_method",
"int", "int1", "int2", "int3", "int4", "int8", "integer", "interval",
"into", "io_after_gtids", "is", "isolation", "iterate", "invoker", "join",
"json", "json_table", "key", "keys", "keyspaces", "key_block_size", "kill", "lag",
"language", "last", "last_value", "last_insert_id", "lateral", "lead", "leading",
"leave", "left", "less", "level", "like", "limit", "linear", "lines",
"linestring", "load", "local", "localtime", "localtimestamp", "lock", "logs",
"long", "longblob", "longtext", "loop", "low_priority", "manifest",
"master_bind", "match", "max_rows", "maxvalue", "mediumblob", "mediumint",
"mediumtext", "memory", "merge", "microsecond", "middleint", "min_rows", "minute",
"minute_microsecond", "minute_second", "mod", "mode", "modify", "modifies",
"multilinestring", "multipoint", "multipolygon", "month", "name",
"names", "natural", "nchar", "next", "no", "none", "not", "no_write_to_binlog",
"nth_value", "ntile", "null", "numeric", "of", "off", "offset", "on",
"only", "open", "optimize", "optimizer_costs", "option", "optionally",
"or", "order", "out", "outer", "outfile", "over", "overwrite", "pack_keys",
"parser", "partition", "partitioning", "password", "percent_rank", "plugins",
"point", "polygon", "precision", "primary", "privileges", "processlist",
"procedure", "query", "quarter", "range", "rank", "read", "reads", "read_write",
"real", "rebuild", "recursive", "redundant", "references", "regexp", "relay",
"release", "remove", "rename", "reorganize", "repair", "repeat", "repeatable",
"replace", "require", "resignal", "restrict", "return", "retry", "revert",
"revoke", "right", "rlike", "rollback", "row", "row_format", "row_number",
"rows", "s3", "savepoint", "schema", "schemas", "second", "second_microsecond",
"security", "select", "sensitive", "separator", "sequence", "serializable",
"session", "set", "share", "shared", "show", "signal", "signed", "slow",
"smallint", "spatial", "specific", "sql", "sqlexception", "sqlstate",
"sqlwarning", "sql_big_result", "sql_cache", "sql_calc_found_rows",
"sql_no_cache", "sql_small_result", "ssl", "start", "starting",
"stats_auto_recalc", "stats_persistent", "stats_sample_pages", "status",
"storage", "stored", "straight_join", "stream", "system", "vstream",
"table", "tables", "tablespace", "temporary", "temptable", "terminated",
"text", "than", "then", "time", "timestamp", "timestampadd", "timestampdiff",
"tinyblob", "tinyint", "tinytext", "to", "trailing", "transaction", "tree",
"traditional", "trigger", "triggers", "true", "truncate", "uncommitted",
"undefined", "undo", "union", "unique", "unlock", "unsigned", "update",
"upgrade", "usage", "use", "user", "user_resources", "using", "utc_date",
"utc_time", "utc_timestamp", "validation", "values", "variables", "varbinary",
"varchar", "varcharacter", "varying", "vgtid_executed", "virtual", "vindex",
"vindexes", "view", "vitess", "vitess_keyspaces", "vitess_metadata",
"vitess_migration", "vitess_migrations", "vitess_replication_status",
"vitess_shards", "vitess_tablets", "vschema", "warnings", "when",
"where", "while", "window", "with", "without", "work", "write", "xor",
"year", "year_month", "zerofill",
}
// Keywords that could get an additional keyword
var needCustomString = []string{
"DISTINCTROW", "FROM", // Select keywords:
"GROUP BY", "HAVING", "WINDOW",
"FOR",
"ORDER BY", "LIMIT",
"INTO", "PARTITION", "AS", // Insert Keywords:
"ON DUPLICATE KEY UPDATE",
"WHERE", "LIMIT", // Delete keywords
"INFILE", "INTO TABLE", "CHARACTER SET", // Load keywords
"TERMINATED BY", "ENCLOSED BY",
"ESCAPED BY", "STARTING BY",
"TERMINATED BY", "STARTING BY",
"IGNORE",
"VALUE", "VALUES", // Replace tokens
"SET", // Update tokens
"ENGINE =", // Drop tokens
"DEFINER =", "ON SCHEDULE", "RENAME TO", // Alter tokens
"COMMENT", "DO", "INITIAL_SIZE = ", "OPTIONS",
}
var alterTableTokens = [][]string{
{"CUSTOM_FUZZ_STRING"},
{"CUSTOM_ALTTER_TABLE_OPTIONS"},
{"PARTITION_OPTIONS_FOR_ALTER_TABLE"},
}
var alterTokens = [][]string{
{
"DATABASE", "SCHEMA", "DEFINER = ", "EVENT", "FUNCTION", "INSTANCE",
"LOGFILE GROUP", "PROCEDURE", "SERVER",
},
{"CUSTOM_FUZZ_STRING"},
{
"ON SCHEDULE", "ON COMPLETION PRESERVE", "ON COMPLETION NOT PRESERVE",
"ADD UNDOFILE", "OPTIONS",
},
{"RENAME TO", "INITIAL_SIZE = "},
{"ENABLE", "DISABLE", "DISABLE ON SLAVE", "ENGINE"},
{"COMMENT"},
{"DO"},
}
var setTokens = [][]string{
{"CHARACTER SET", "CHARSET", "CUSTOM_FUZZ_STRING", "NAMES"},
{"CUSTOM_FUZZ_STRING", "DEFAULT", "="},
{"CUSTOM_FUZZ_STRING"},
}
var dropTokens = [][]string{
{"TEMPORARY", "UNDO"},
{
"DATABASE", "SCHEMA", "EVENT", "INDEX", "LOGFILE GROUP",
"PROCEDURE", "FUNCTION", "SERVER", "SPATIAL REFERENCE SYSTEM",
"TABLE", "TABLESPACE", "TRIGGER", "VIEW",
},
{"IF EXISTS"},
{"CUSTOM_FUZZ_STRING"},
{"ON", "ENGINE = ", "RESTRICT", "CASCADE"},
}
var renameTokens = [][]string{
{"TABLE"},
{"CUSTOM_FUZZ_STRING"},
{"TO"},
{"CUSTOM_FUZZ_STRING"},
}
var truncateTokens = [][]string{
{"TABLE"},
{"CUSTOM_FUZZ_STRING"},
}
var createTokens = [][]string{
{"OR REPLACE", "TEMPORARY", "UNDO"}, // For create spatial reference system
{
"UNIQUE", "FULLTEXT", "SPATIAL", "ALGORITHM = UNDEFINED", "ALGORITHM = MERGE",
"ALGORITHM = TEMPTABLE",
},
{
"DATABASE", "SCHEMA", "EVENT", "FUNCTION", "INDEX", "LOGFILE GROUP",
"PROCEDURE", "SERVER", "SPATIAL REFERENCE SYSTEM", "TABLE", "TABLESPACE",
"TRIGGER", "VIEW",
},
{"IF NOT EXISTS"},
{"CUSTOM_FUZZ_STRING"},
}
/*
// For future use.
var updateTokens = [][]string{
{"LOW_PRIORITY"},
{"IGNORE"},
{"SET"},
{"WHERE"},
{"ORDER BY"},
{"LIMIT"},
}
*/
var replaceTokens = [][]string{
{"LOW_PRIORITY", "DELAYED"},
{"INTO"},
{"PARTITION"},
{"CUSTOM_FUZZ_STRING"},
{"VALUES", "VALUE"},
}
var loadTokens = [][]string{
{"DATA"},
{"LOW_PRIORITY", "CONCURRENT", "LOCAL"},
{"INFILE"},
{"REPLACE", "IGNORE"},
{"INTO TABLE"},
{"PARTITION"},
{"CHARACTER SET"},
{"FIELDS", "COLUMNS"},
{"TERMINATED BY"},
{"OPTIONALLY"},
{"ENCLOSED BY"},
{"ESCAPED BY"},
{"LINES"},
{"STARTING BY"},
{"TERMINATED BY"},
{"IGNORE"},
{"LINES", "ROWS"},
{"CUSTOM_FUZZ_STRING"},
}
// These Are everything that comes after "INSERT"
var insertTokens = [][]string{
{"LOW_PRIORITY", "DELAYED", "HIGH_PRIORITY", "IGNORE"},
{"INTO"},
{"PARTITION"},
{"CUSTOM_FUZZ_STRING"},
{"AS"},
{"ON DUPLICATE KEY UPDATE"},
}
// These are everything that comes after "SELECT"
var selectTokens = [][]string{
{"*", "CUSTOM_FUZZ_STRING", "DISTINCTROW"},
{"HIGH_PRIORITY"},
{"STRAIGHT_JOIN"},
{"SQL_SMALL_RESULT", "SQL_BIG_RESULT", "SQL_BUFFER_RESULT"},
{"SQL_NO_CACHE", "SQL_CALC_FOUND_ROWS"},
{"CUSTOM_FUZZ_STRING"},
{"FROM"},
{"WHERE"},
{"GROUP BY"},
{"HAVING"},
{"WINDOW"},
{"ORDER BY"},
{"LIMIT"},
{"CUSTOM_FUZZ_STRING"},
{"FOR"},
}
// These are everything that comes after "DELETE"
var deleteTokens = [][]string{
{"LOW_PRIORITY", "QUICK", "IGNORE", "FROM", "AS"},
{"PARTITION"},
{"WHERE"},
{"ORDER BY"},
{"LIMIT"},
}
var alter_table_options = []string{
"ADD", "COLUMN", "FIRST", "AFTER", "INDEX", "KEY", "FULLTEXT", "SPATIAL",
"CONSTRAINT", "UNIQUE", "FOREIGN KEY", "CHECK", "ENFORCED", "DROP", "ALTER",
"NOT", "INPLACE", "COPY", "SET", "VISIBLE", "INVISIBLE", "DEFAULT", "CHANGE",
"CHARACTER SET", "COLLATE", "DISABLE", "ENABLE", "KEYS", "TABLESPACE", "LOCK",
"FORCE", "MODIFY", "SHARED", "EXCLUSIVE", "NONE", "ORDER BY", "RENAME COLUMN",
"AS", "=", "ASC", "DESC", "WITH", "WITHOUT", "VALIDATION", "ADD PARTITION",
"DROP PARTITION", "DISCARD PARTITION", "IMPORT PARTITION", "TRUNCATE PARTITION",
"COALESCE PARTITION", "REORGANIZE PARTITION", "EXCHANGE PARTITION",
"ANALYZE PARTITION", "CHECK PARTITION", "OPTIMIZE PARTITION", "REBUILD PARTITION",
"REPAIR PARTITION", "REMOVE PARTITIONING", "USING", "BTREE", "HASH", "COMMENT",
"KEY_BLOCK_SIZE", "WITH PARSER", "AUTOEXTEND_SIZE", "AUTO_INCREMENT", "AVG_ROW_LENGTH",
"CHECKSUM", "INSERT_METHOD", "ROW_FORMAT", "DYNAMIC", "FIXED", "COMPRESSED", "REDUNDANT",
"COMPACT", "SECONDARY_ENGINE_ATTRIBUTE", "STATS_AUTO_RECALC", "STATS_PERSISTENT",
"STATS_SAMPLE_PAGES", "ZLIB", "LZ4", "ENGINE_ATTRIBUTE", "KEY_BLOCK_SIZE", "MAX_ROWS",
"MIN_ROWS", "PACK_KEYS", "PASSWORD", "COMPRESSION", "CONNECTION", "DIRECTORY",
"DELAY_KEY_WRITE", "ENCRYPTION", "STORAGE", "DISK", "MEMORY", "UNION",
}
// Creates an 'alter table' statement. 'alter table' is an exception
// in that it has its own function. The majority of statements
// are created by 'createStmt()'.
func createAlterTableStmt(f *ConsumeFuzzer) (string, error) {
maxArgs, err := f.GetInt()
if err != nil {
return "", err
}
maxArgs = maxArgs % 30
if maxArgs == 0 {
return "", fmt.Errorf("could not create alter table stmt")
}
var stmt strings.Builder
stmt.WriteString("ALTER TABLE ")
for i := 0; i < maxArgs; i++ {
// Calculate if we get existing token or custom string
tokenType, err := f.GetInt()
if err != nil {
return "", err
}
if tokenType%4 == 1 {
customString, err := f.GetString()
if err != nil {
return "", err
}
stmt.WriteString(" " + customString)
} else {
tokenIndex, err := f.GetInt()
if err != nil {
return "", err
}
stmt.WriteString(" " + alter_table_options[tokenIndex%len(alter_table_options)])
}
}
return stmt.String(), nil
}
func chooseToken(tokens []string, f *ConsumeFuzzer) (string, error) {
index, err := f.GetInt()
if err != nil {
return "", err
}
var token strings.Builder
token.WriteString(tokens[index%len(tokens)])
if token.String() == "CUSTOM_FUZZ_STRING" {
customFuzzString, err := f.GetString()
if err != nil {
return "", err
}
return customFuzzString, nil
}
// Check if token requires an argument
if containsString(needCustomString, token.String()) {
customFuzzString, err := f.GetString()
if err != nil {
return "", err
}
token.WriteString(" " + customFuzzString)
}
return token.String(), nil
}
var stmtTypes = map[string][][]string{
"DELETE": deleteTokens,
"INSERT": insertTokens,
"SELECT": selectTokens,
"LOAD": loadTokens,
"REPLACE": replaceTokens,
"CREATE": createTokens,
"DROP": dropTokens,
"RENAME": renameTokens,
"TRUNCATE": truncateTokens,
"SET": setTokens,
"ALTER": alterTokens,
"ALTER TABLE": alterTableTokens, // ALTER TABLE has its own set of tokens
}
var stmtTypeEnum = map[int]string{
0: "DELETE",
1: "INSERT",
2: "SELECT",
3: "LOAD",
4: "REPLACE",
5: "CREATE",
6: "DROP",
7: "RENAME",
8: "TRUNCATE",
9: "SET",
10: "ALTER",
11: "ALTER TABLE",
}
func createStmt(f *ConsumeFuzzer) (string, error) {
stmtIndex, err := f.GetInt()
if err != nil {
return "", err
}
stmtIndex = stmtIndex % len(stmtTypes)
queryType := stmtTypeEnum[stmtIndex]
tokens := stmtTypes[queryType]
// We have custom creator for ALTER TABLE
if queryType == "ALTER TABLE" {
query, err := createAlterTableStmt(f)
if err != nil {
return "", err
}
return query, nil
}
// Here we are creating a query that is not
// an 'alter table' query. For available
// queries, see "stmtTypes"
// First specify the first query keyword:
var query strings.Builder
query.WriteString(queryType)
// Next create the args for the
queryArgs, err := createStmtArgs(tokens, f)
if err != nil {
return "", err
}
query.WriteString(" " + queryArgs)
return query.String(), nil
}
// Creates the arguments of a statements. In a select statement
// that would be everything after "select".
func createStmtArgs(tokenslice [][]string, f *ConsumeFuzzer) (string, error) {
var query, token strings.Builder
// We go through the tokens in the tokenslice,
// create the respective token and add it to
// "query"
for _, tokens := range tokenslice {
// For extra randomization, the fuzzer can
// choose to not include this token.
includeThisToken, err := f.GetBool()
if err != nil {
return "", err
}
if !includeThisToken {
continue
}
// There may be several tokens to choose from:
if len(tokens) > 1 {
chosenToken, err := chooseToken(tokens, f)
if err != nil {
return "", err
}
query.WriteString(" " + chosenToken)
} else {
token.WriteString(tokens[0])
// In case the token is "CUSTOM_FUZZ_STRING"
// we will then create a non-structured string
if token.String() == "CUSTOM_FUZZ_STRING" {
customFuzzString, err := f.GetString()
if err != nil {
return "", err
}
query.WriteString(" " + customFuzzString)
continue
}
// Check if token requires an argument.
// Tokens that take an argument can be found
// in 'needCustomString'. If so, we add a
// non-structured string to the token.
if containsString(needCustomString, token.String()) {
customFuzzString, err := f.GetString()
if err != nil {
return "", err
}
token.WriteString(fmt.Sprintf(" %s", customFuzzString))
}
query.WriteString(fmt.Sprintf(" %s", token.String()))
}
}
return query.String(), nil
}
// Creates a semi-structured query. It creates a string
// that is a combination of the keywords and random strings.
func createQuery(f *ConsumeFuzzer) (string, error) {
queryLen, err := f.GetInt()
if err != nil {
return "", err
}
maxLen := queryLen % 60
if maxLen == 0 {
return "", fmt.Errorf("could not create a query")
}
var query strings.Builder
for i := 0; i < maxLen; i++ {
// Get a new token:
useKeyword, err := f.GetBool()
if err != nil {
return "", err
}
if useKeyword {
keyword, err := getKeyword(f)
if err != nil {
return "", err
}
query.WriteString(" " + keyword)
} else {
customString, err := f.GetString()
if err != nil {
return "", err
}
query.WriteString(" " + customString)
}
}
if query.String() == "" {
return "", fmt.Errorf("could not create a query")
}
return query.String(), nil
}
// GetSQLString is the API that users interact with.
//
// Usage:
//
// f := NewConsumer(data)
// sqlString, err := f.GetSQLString()
func (f *ConsumeFuzzer) GetSQLString() (string, error) {
var query string
veryStructured, err := f.GetBool()
if err != nil {
return "", err
}
if veryStructured {
query, err = createStmt(f)
if err != nil {
return "", err
}
} else {
query, err = createQuery(f)
if err != nil {
return "", err
}
}
return query, nil
}

21
vendor/github.com/Microsoft/hcsshim/LICENSE generated vendored Normal file
View File

@@ -0,0 +1,21 @@
The MIT License (MIT)
Copyright (c) 2015 Microsoft
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

59
vendor/github.com/Microsoft/hcsshim/osversion/osversion_windows.go generated vendored Normal file
View File

@@ -0,0 +1,59 @@
package osversion
import (
"fmt"
"sync"
"golang.org/x/sys/windows"
)
// OSVersion is a wrapper for Windows version information
// https://msdn.microsoft.com/en-us/library/windows/desktop/ms724439(v=vs.85).aspx
type OSVersion struct {
Version uint32
MajorVersion uint8
MinorVersion uint8
Build uint16
}
var (
osv OSVersion
once sync.Once
)
// Get gets the operating system version on Windows.
// The calling application must be manifested to get the correct version information.
func Get() OSVersion {
once.Do(func() {
var err error
osv = OSVersion{}
osv.Version, err = windows.GetVersion()
if err != nil {
// GetVersion never fails.
panic(err)
}
osv.MajorVersion = uint8(osv.Version & 0xFF)
osv.MinorVersion = uint8(osv.Version >> 8 & 0xFF)
osv.Build = uint16(osv.Version >> 16)
})
return osv
}
// Build gets the build-number on Windows
// The calling application must be manifested to get the correct version information.
func Build() uint16 {
return Get().Build
}
// String returns the OSVersion formatted as a string. It implements the
// [fmt.Stringer] interface.
func (osv OSVersion) String() string {
return fmt.Sprintf("%d.%d.%d", osv.MajorVersion, osv.MinorVersion, osv.Build)
}
// ToString returns the OSVersion formatted as a string.
//
// Deprecated: use [OSVersion.String].
func (osv OSVersion) ToString() string {
return osv.String()
}

35
vendor/github.com/Microsoft/hcsshim/osversion/platform_compat_windows.go generated vendored Normal file
View File

@@ -0,0 +1,35 @@
package osversion
// List of stable ABI compliant ltsc releases
// Note: List must be sorted in ascending order
var compatLTSCReleases = []uint16{
V21H2Server,
}
// CheckHostAndContainerCompat checks if given host and container
// OS versions are compatible.
// It includes support for stable ABI compliant versions as well.
// Every release after WS 2022 will support the previous ltsc
// container image. Stable ABI is in preview mode for windows 11 client.
// Refer: https://learn.microsoft.com/en-us/virtualization/windowscontainers/deploy-containers/version-compatibility?tabs=windows-server-2022%2Cwindows-10#windows-server-host-os-compatibility
func CheckHostAndContainerCompat(host, ctr OSVersion) bool {
// check major minor versions of host and guest
if host.MajorVersion != ctr.MajorVersion ||
host.MinorVersion != ctr.MinorVersion {
return false
}
// If host is < WS 2022, exact version match is required
if host.Build < V21H2Server {
return host.Build == ctr.Build
}
var supportedLtscRelease uint16
for i := len(compatLTSCReleases) - 1; i >= 0; i-- {
if host.Build >= compatLTSCReleases[i] {
supportedLtscRelease = compatLTSCReleases[i]
break
}
}
return ctr.Build >= supportedLtscRelease && ctr.Build <= host.Build
}

84
vendor/github.com/Microsoft/hcsshim/osversion/windowsbuilds.go generated vendored Normal file
View File

@@ -0,0 +1,84 @@
package osversion
// Windows Client and Server build numbers.
//
// See:
// https://learn.microsoft.com/en-us/windows/release-health/release-information
// https://learn.microsoft.com/en-us/windows/release-health/windows-server-release-info
// https://learn.microsoft.com/en-us/windows/release-health/windows11-release-information
const (
// RS1 (version 1607, codename "Redstone 1") corresponds to Windows Server
// 2016 (ltsc2016) and Windows 10 (Anniversary Update).
RS1 = 14393
// V1607 (version 1607, codename "Redstone 1") is an alias for [RS1].
V1607 = RS1
// LTSC2016 (Windows Server 2016) is an alias for [RS1].
LTSC2016 = RS1
// RS2 (version 1703, codename "Redstone 2") was a client-only update, and
// corresponds to Windows 10 (Creators Update).
RS2 = 15063
// V1703 (version 1703, codename "Redstone 2") is an alias for [RS2].
V1703 = RS2
// RS3 (version 1709, codename "Redstone 3") corresponds to Windows Server
// 1709 (Semi-Annual Channel (SAC)), and Windows 10 (Fall Creators Update).
RS3 = 16299
// V1709 (version 1709, codename "Redstone 3") is an alias for [RS3].
V1709 = RS3
// RS4 (version 1803, codename "Redstone 4") corresponds to Windows Server
// 1803 (Semi-Annual Channel (SAC)), and Windows 10 (April 2018 Update).
RS4 = 17134
// V1803 (version 1803, codename "Redstone 4") is an alias for [RS4].
V1803 = RS4
// RS5 (version 1809, codename "Redstone 5") corresponds to Windows Server
// 2019 (ltsc2019), and Windows 10 (October 2018 Update).
RS5 = 17763
// V1809 (version 1809, codename "Redstone 5") is an alias for [RS5].
V1809 = RS5
// LTSC2019 (Windows Server 2019) is an alias for [RS5].
LTSC2019 = RS5
// V19H1 (version 1903, codename 19H1) corresponds to Windows Server 1903 (semi-annual
// channel).
V19H1 = 18362
// V1903 (version 1903) is an alias for [V19H1].
V1903 = V19H1
// V19H2 (version 1909, codename 19H2) corresponds to Windows Server 1909 (semi-annual
// channel).
V19H2 = 18363
// V1909 (version 1909) is an alias for [V19H2].
V1909 = V19H2
// V20H1 (version 2004, codename 20H1) corresponds to Windows Server 2004 (semi-annual
// channel).
V20H1 = 19041
// V2004 (version 2004) is an alias for [V20H1].
V2004 = V20H1
// V20H2 corresponds to Windows Server 20H2 (semi-annual channel).
V20H2 = 19042
// V21H1 corresponds to Windows Server 21H1 (semi-annual channel).
V21H1 = 19043
// V21H2Win10 corresponds to Windows 10 (November 2021 Update).
V21H2Win10 = 19044
// V21H2Server corresponds to Windows Server 2022 (ltsc2022).
V21H2Server = 20348
// LTSC2022 (Windows Server 2022) is an alias for [V21H2Server]
LTSC2022 = V21H2Server
// V21H2Win11 corresponds to Windows 11 (original release).
V21H2Win11 = 22000
// V22H2Win10 corresponds to Windows 10 (2022 Update).
V22H2Win10 = 19045
// V22H2Win11 corresponds to Windows 11 (2022 Update).
V22H2Win11 = 22621
)

2
vendor/github.com/containerd/containerd/archive/compression/compression.go generated vendored
View File

@@ -25,12 +25,12 @@ import (
"fmt"
"io"
"os"
"os/exec"
"strconv"
"sync"
"github.com/containerd/containerd/log"
"github.com/klauspost/compress/zstd"
exec "golang.org/x/sys/execabs"
)
type (

28
vendor/github.com/containerd/containerd/archive/compression/compression_fuzzer.go generated vendored Normal file
View File

@@ -0,0 +1,28 @@
//go:build gofuzz
/*
Copyright The containerd Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package compression
import (
"bytes"
)
func FuzzDecompressStream(data []byte) int {
_, _ = DecompressStream(bytes.NewReader(data))
return 1
}

87
vendor/github.com/containerd/containerd/content/content.go generated vendored
View File

@@ -25,6 +25,26 @@ import (
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
)
// Store combines the methods of content-oriented interfaces into a set that
// are commonly provided by complete implementations.
//
// Overall content lifecycle:
// - Ingester is used to initiate a write operation (aka ingestion)
// - IngestManager is used to manage (e.g. list, abort) active ingestions
// - Once an ingestion is complete (see Writer.Commit), Provider is used to
// query a single piece of content by its digest
// - Manager is used to manage (e.g. list, delete) previously committed content
//
// Note that until ingestion is complete, its content is not visible through
// Provider or Manager. Once ingestion is complete, it is no longer exposed
// through IngestManager.
type Store interface {
Manager
Provider
IngestManager
Ingester
}
// ReaderAt extends the standard io.ReaderAt interface with reporting of Size and io.Closer
type ReaderAt interface {
io.ReaderAt
@@ -42,14 +62,31 @@ type Provider interface {
// Ingester writes content
type Ingester interface {
// Some implementations require WithRef to be included in opts.
// Writer initiates a writing operation (aka ingestion). A single ingestion
// is uniquely identified by its ref, provided using a WithRef option.
// Writer can be called multiple times with the same ref to access the same
// ingestion.
// Once all the data is written, use Writer.Commit to complete the ingestion.
Writer(ctx context.Context, opts ...WriterOpt) (Writer, error)
}
// IngestManager provides methods for managing ingestions. An ingestion is a
// not-yet-complete writing operation initiated using Ingester and identified
// by a ref string.
type IngestManager interface {
// Status returns the status of the provided ref.
Status(ctx context.Context, ref string) (Status, error)
// ListStatuses returns the status of any active ingestions whose ref match
// the provided regular expression. If empty, all active ingestions will be
// returned.
ListStatuses(ctx context.Context, filters ...string) ([]Status, error)
// Abort completely cancels the ingest operation targeted by ref.
Abort(ctx context.Context, ref string) error
}
// Info holds content specific information
//
// TODO(stevvooe): Consider a very different name for this struct. Info is way
// to general. It also reads very weird in certain context, like pluralization.
type Info struct {
Digest digest.Digest
Size int64
@@ -58,7 +95,7 @@ type Info struct {
Labels map[string]string
}
// Status of a content operation
// Status of a content operation (i.e. an ingestion)
type Status struct {
Ref string
Offset int64
@@ -71,12 +108,23 @@ type Status struct {
// WalkFunc defines the callback for a blob walk.
type WalkFunc func(Info) error
// Manager provides methods for inspecting, listing and removing content.
type Manager interface {
// InfoReaderProvider provides both info and reader for the specific content.
type InfoReaderProvider interface {
InfoProvider
Provider
}
// InfoProvider provides info for content inspection.
type InfoProvider interface {
// Info will return metadata about content available in the content store.
//
// If the content is not present, ErrNotFound will be returned.
Info(ctx context.Context, dgst digest.Digest) (Info, error)
}
// Manager provides methods for inspecting, listing and removing content.
type Manager interface {
InfoProvider
// Update updates mutable information related to content.
// If one or more fieldpaths are provided, only those
@@ -94,21 +142,7 @@ type Manager interface {
Delete(ctx context.Context, dgst digest.Digest) error
}
// IngestManager provides methods for managing ingests.
type IngestManager interface {
// Status returns the status of the provided ref.
Status(ctx context.Context, ref string) (Status, error)
// ListStatuses returns the status of any active ingestions whose ref match the
// provided regular expression. If empty, all active ingestions will be
// returned.
ListStatuses(ctx context.Context, filters ...string) ([]Status, error)
// Abort completely cancels the ingest operation targeted by ref.
Abort(ctx context.Context, ref string) error
}
// Writer handles the write of content into a content store
// Writer handles writing of content into a content store
type Writer interface {
// Close closes the writer, if the writer has not been
// committed this allows resuming or aborting.
@@ -131,15 +165,6 @@ type Writer interface {
Truncate(size int64) error
}
// Store combines the methods of content-oriented interfaces into a set that
// are commonly provided by complete implementations.
type Store interface {
Manager
Provider
IngestManager
Ingester
}
// Opt is used to alter the mutable properties of content
type Opt func(*Info) error

29
vendor/github.com/containerd/containerd/content/helpers.go generated vendored
View File

@@ -21,12 +21,12 @@ import (
"errors"
"fmt"
"io"
"math/rand"
"sync"
"time"
"github.com/containerd/containerd/errdefs"
"github.com/containerd/containerd/log"
"github.com/containerd/containerd/pkg/randutil"
"github.com/opencontainers/go-digest"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
)
@@ -43,16 +43,26 @@ var bufPool = sync.Pool{
},
}
type reader interface {
Reader() io.Reader
}
// NewReader returns a io.Reader from a ReaderAt
func NewReader(ra ReaderAt) io.Reader {
rd := io.NewSectionReader(ra, 0, ra.Size())
return rd
if rd, ok := ra.(reader); ok {
return rd.Reader()
}
return io.NewSectionReader(ra, 0, ra.Size())
}
// ReadBlob retrieves the entire contents of the blob from the provider.
//
// Avoid using this for large blobs, such as layers.
func ReadBlob(ctx context.Context, provider Provider, desc ocispec.Descriptor) ([]byte, error) {
if int64(len(desc.Data)) == desc.Size && digest.FromBytes(desc.Data) == desc.Digest {
return desc.Data, nil
}
ra, err := provider.ReaderAt(ctx, desc)
if err != nil {
return nil, err
@@ -113,7 +123,7 @@ func OpenWriter(ctx context.Context, cs Ingester, opts ...WriterOpt) (Writer, er
// error or abort. Requires asserting for an ingest manager
select {
case <-time.After(time.Millisecond * time.Duration(rand.Intn(retry))):
case <-time.After(time.Millisecond * time.Duration(randutil.Intn(retry))):
if retry < 2048 {
retry = retry << 1
}
@@ -322,3 +332,14 @@ func copyWithBuffer(dst io.Writer, src io.Reader) (written int64, err error) {
}
return
}
// Exists returns whether an attempt to access the content would not error out
// with an ErrNotFound error. It will return an encountered error if it was
// different than ErrNotFound.
func Exists(ctx context.Context, provider InfoProvider, desc ocispec.Descriptor) (bool, error) {
_, err := provider.Info(ctx, desc.Digest)
if errdefs.IsNotFound(err) {
return false, nil
}
return err == nil, err
}

76
vendor/github.com/containerd/containerd/content/local/content_local_fuzzer.go generated vendored Normal file
View File

@@ -0,0 +1,76 @@
//go:build gofuzz
/*
Copyright The containerd Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package local
import (
"bufio"
"bytes"
"context"
_ "crypto/sha256"
"io"
"testing"
"github.com/opencontainers/go-digest"
"github.com/containerd/containerd/content"
)
func FuzzContentStoreWriter(data []byte) int {
t := &testing.T{}
ctx := context.Background()
ctx, _, cs, cleanup := contentStoreEnv(t)
defer cleanup()
cw, err := cs.Writer(ctx, content.WithRef("myref"))
if err != nil {
return 0
}
if err := cw.Close(); err != nil {
return 0
}
// reopen, so we can test things
cw, err = cs.Writer(ctx, content.WithRef("myref"))
if err != nil {
return 0
}
err = checkCopyFuzz(int64(len(data)), cw, bufio.NewReader(io.NopCloser(bytes.NewReader(data))))
if err != nil {
return 0
}
expected := digest.FromBytes(data)
if err = cw.Commit(ctx, int64(len(data)), expected); err != nil {
return 0
}
return 1
}
func checkCopyFuzz(size int64, dst io.Writer, src io.Reader) error {
nn, err := io.Copy(dst, src)
if err != nil {
return err
}
if nn != size {
return err
}
return nil
}

5
vendor/github.com/containerd/containerd/content/local/readerat.go generated vendored
View File

@@ -18,6 +18,7 @@ package local
import (
"fmt"
"io"
"os"
"github.com/containerd/containerd/content"
@@ -65,3 +66,7 @@ func (ra sizeReaderAt) Size() int64 {
func (ra sizeReaderAt) Close() error {
return ra.fp.Close()
}
func (ra sizeReaderAt) Reader() io.Reader {
return io.LimitReader(ra.fp, ra.size)
}

13
vendor/github.com/containerd/containerd/content/local/store.go generated vendored
View File

@@ -20,7 +20,6 @@ import (
"context"
"fmt"
"io"
"math/rand"
"os"
"path/filepath"
"strconv"
@@ -32,9 +31,10 @@ import (
"github.com/containerd/containerd/errdefs"
"github.com/containerd/containerd/filters"
"github.com/containerd/containerd/log"
"github.com/containerd/containerd/pkg/randutil"
"github.com/sirupsen/logrus"
digest "github.com/opencontainers/go-digest"
"github.com/opencontainers/go-digest"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
)
@@ -262,7 +262,7 @@ func (s *store) Walk(ctx context.Context, fn content.WalkFunc, fs ...string) err
return nil
}
dgst := digest.NewDigestFromHex(alg.String(), filepath.Base(path))
dgst := digest.NewDigestFromEncoded(alg, filepath.Base(path))
if err := dgst.Validate(); err != nil {
// log error but don't report
log.L.WithError(err).WithField("path", path).Error("invalid digest for blob path")
@@ -473,7 +473,7 @@ func (s *store) Writer(ctx context.Context, opts ...content.WriterOpt) (content.
lockErr = nil
break
}
time.Sleep(time.Millisecond * time.Duration(rand.Intn(1<<count)))
time.Sleep(time.Millisecond * time.Duration(randutil.Intn(1<<count)))
}
if lockErr != nil {
@@ -505,6 +505,7 @@ func (s *store) resumeStatus(ref string, total int64, digester digest.Digester)
return status, fmt.Errorf("provided total differs from status: %v != %v", total, status.Total)
}
//nolint:dupword
// TODO(stevvooe): slow slow slow!!, send to goroutine or use resumable hashes
fp, err := os.Open(data)
if err != nil {
@@ -628,14 +629,14 @@ func (s *store) blobPath(dgst digest.Digest) (string, error) {
return "", fmt.Errorf("cannot calculate blob path from invalid digest: %v: %w", err, errdefs.ErrInvalidArgument)
}
return filepath.Join(s.root, "blobs", dgst.Algorithm().String(), dgst.Hex()), nil
return filepath.Join(s.root, "blobs", dgst.Algorithm().String(), dgst.Encoded()), nil
}
func (s *store) ingestRoot(ref string) string {
// we take a digest of the ref to keep the ingest paths constant length.
// Note that this is not the current or potential digest of incoming content.
dgst := digest.FromString(ref)
return filepath.Join(s.root, "ingest", dgst.Hex())
return filepath.Join(s.root, "ingest", dgst.Encoded())
}
// ingestPaths are returned. The paths are the following:

1
vendor/github.com/containerd/containerd/content/local/store_bsd.go generated vendored
View File

@@ -1,5 +1,4 @@
//go:build darwin || freebsd || netbsd
// +build darwin freebsd netbsd
/*
Copyright The containerd Authors.

1
vendor/github.com/containerd/containerd/content/local/store_openbsd.go generated vendored
View File

@@ -1,5 +1,4 @@
//go:build openbsd
// +build openbsd
/*
Copyright The containerd Authors.

1
vendor/github.com/containerd/containerd/content/local/store_unix.go generated vendored
View File

@@ -1,5 +1,4 @@
//go:build linux || solaris
// +build linux solaris
/*
Copyright The containerd Authors.

38
vendor/github.com/containerd/containerd/content/local/test_helper.go generated vendored Normal file
View File

@@ -0,0 +1,38 @@
/*
Copyright The containerd Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package local
import (
"context"
"testing"
"github.com/containerd/containerd/content"
)
func contentStoreEnv(t testing.TB) (context.Context, string, content.Store, func()) {
tmpdir := t.TempDir()
cs, err := NewStore(tmpdir)
if err != nil {
t.Fatal(err)
}
ctx, cancel := context.WithCancel(context.Background())
return ctx, tmpdir, cs, func() {
cancel()
}
}

4
vendor/github.com/containerd/containerd/images/image.go generated vendored
View File

@@ -138,7 +138,7 @@ type platformManifest struct {
// TODO(stevvooe): This violates the current platform agnostic approach to this
// package by returning a specific manifest type. We'll need to refactor this
// to return a manifest descriptor or decide that we want to bring the API in
// this direction because this abstraction is not needed.`
// this direction because this abstraction is not needed.
func Manifest(ctx context.Context, provider content.Provider, image ocispec.Descriptor, platform platforms.MatchComparer) (ocispec.Manifest, error) {
var (
limit = 1
@@ -311,7 +311,7 @@ func Check(ctx context.Context, provider content.Provider, image ocispec.Descrip
return false, nil, nil, nil, fmt.Errorf("failed to check image %v: %w", image.Digest, err)
}
// TODO(stevvooe): It is possible that referenced conponents could have
// TODO(stevvooe): It is possible that referenced components could have
// children, but this is rare. For now, we ignore this and only verify
// that manifest components are present.
required = append([]ocispec.Descriptor{mfst.Config}, mfst.Layers...)

21
vendor/github.com/containerd/containerd/images/labels.go generated vendored Normal file
View File

@@ -0,0 +1,21 @@
/*
Copyright The containerd Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package images
const (
ConvertedDockerSchema1LabelKey = "io.containerd.image/converted-docker-schema1"
)

31
vendor/github.com/containerd/containerd/images/mediatypes.go generated vendored
View File

@@ -38,7 +38,9 @@ const (
MediaTypeDockerSchema2Config = "application/vnd.docker.container.image.v1+json"
MediaTypeDockerSchema2Manifest = "application/vnd.docker.distribution.manifest.v2+json"
MediaTypeDockerSchema2ManifestList = "application/vnd.docker.distribution.manifest.list.v2+json"
// Checkpoint/Restore Media Types
MediaTypeContainerd1Checkpoint = "application/vnd.containerd.container.criu.checkpoint.criu.tar"
MediaTypeContainerd1CheckpointPreDump = "application/vnd.containerd.container.criu.checkpoint.predump.tar"
MediaTypeContainerd1Resource = "application/vnd.containerd.container.resource.tar"
@@ -47,9 +49,12 @@ const (
MediaTypeContainerd1CheckpointOptions = "application/vnd.containerd.container.checkpoint.options.v1+proto"
MediaTypeContainerd1CheckpointRuntimeName = "application/vnd.containerd.container.checkpoint.runtime.name"
MediaTypeContainerd1CheckpointRuntimeOptions = "application/vnd.containerd.container.checkpoint.runtime.options+proto"
// Legacy Docker schema1 manifest
// MediaTypeDockerSchema1Manifest is the legacy Docker schema1 manifest
MediaTypeDockerSchema1Manifest = "application/vnd.docker.distribution.manifest.v1+prettyjws"
// Encypted media types
// Encrypted media types
MediaTypeImageLayerEncrypted = ocispec.MediaTypeImageLayer + "+encrypted"
MediaTypeImageLayerGzipEncrypted = ocispec.MediaTypeImageLayerGzip + "+encrypted"
)
@@ -93,16 +98,23 @@ func DiffCompression(ctx context.Context, mediaType string) (string, error) {
// parseMediaTypes splits the media type into the base type and
// an array of sorted extensions
func parseMediaTypes(mt string) (string, []string) {
func parseMediaTypes(mt string) (mediaType string, suffixes []string) {
if mt == "" {
return "", []string{}
}
mediaType, ext, ok := strings.Cut(mt, "+")
if !ok {
return mediaType, []string{}
}
s := strings.Split(mt, "+")
ext := s[1:]
sort.Strings(ext)
return s[0], ext
// Splitting the extensions following the mediatype "(+)gzip+encrypted".
// We expect this to be a limited list, so add an arbitrary limit (50).
//
// Note that DiffCompression is only using the last element, so perhaps we
// should split on the last "+" only.
suffixes = strings.SplitN(ext, "+", 50)
sort.Strings(suffixes)
return mediaType, suffixes
}
// IsNonDistributable returns true if the media type is non-distributable.
@@ -118,8 +130,7 @@ func IsLayerType(mt string) bool {
}
// Parse Docker media types, strip off any + suffixes first
base, _ := parseMediaTypes(mt)
switch base {
switch base, _ := parseMediaTypes(mt); base {
case MediaTypeDockerSchema2Layer, MediaTypeDockerSchema2LayerGzip,
MediaTypeDockerSchema2LayerForeign, MediaTypeDockerSchema2LayerForeignGzip:
return true

8
vendor/github.com/containerd/containerd/labels/labels.go generated vendored
View File

@@ -19,3 +19,11 @@ package labels
// LabelUncompressed is added to compressed layer contents.
// The value is digest of the uncompressed content.
const LabelUncompressed = "containerd.io/uncompressed"
// LabelSharedNamespace is added to a namespace to allow that namespaces
// contents to be shared.
const LabelSharedNamespace = "containerd.io/namespace.shareable"
// LabelDistributionSource is added to content to indicate its origin.
// e.g., "containerd.io/distribution.source.docker.io=library/redis"
const LabelDistributionSource = "containerd.io/distribution.source"

11
vendor/github.com/containerd/containerd/labels/validate.go generated vendored
View File

@@ -24,15 +24,18 @@ import (
const (
maxSize = 4096
// maximum length of key portion of error message if len of key + len of value > maxSize
keyMaxLen = 64
)
// Validate a label's key and value are under 4096 bytes
func Validate(k, v string) error {
if (len(k) + len(v)) > maxSize {
if len(k) > 10 {
k = k[:10]
total := len(k) + len(v)
if total > maxSize {
if len(k) > keyMaxLen {
k = k[:keyMaxLen]
}
return fmt.Errorf("label key and value greater than maximum size (%d bytes), key: %s: %w", maxSize, k, errdefs.ErrInvalidArgument)
return fmt.Errorf("label key and value length (%d bytes) greater than maximum size (%d bytes), key: %s: %w", total, maxSize, k, errdefs.ErrInvalidArgument)
}
return nil
}

69
vendor/github.com/containerd/containerd/log/context.go generated vendored
View File

@@ -1,69 +0,0 @@
/*
Copyright The containerd Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package log
import (
"context"
"github.com/sirupsen/logrus"
)
var (
// G is an alias for GetLogger.
//
// We may want to define this locally to a package to get package tagged log
// messages.
G = GetLogger
// L is an alias for the standard logger.
L = logrus.NewEntry(logrus.StandardLogger())
)
type (
loggerKey struct{}
)
const (
// RFC3339NanoFixed is time.RFC3339Nano with nanoseconds padded using zeros to
// ensure the formatted time is always the same number of characters.
RFC3339NanoFixed = "2006-01-02T15:04:05.000000000Z07:00"
// TextFormat represents the text logging format
TextFormat = "text"
// JSONFormat represents the JSON logging format
JSONFormat = "json"
)
// WithLogger returns a new context with the provided logger. Use in
// combination with logger.WithField(s) for great effect.
func WithLogger(ctx context.Context, logger *logrus.Entry) context.Context {
e := logger.WithContext(ctx)
return context.WithValue(ctx, loggerKey{}, e)
}
// GetLogger retrieves the current logger from the context. If no logger is
// available, the default logger is returned.
func GetLogger(ctx context.Context) *logrus.Entry {
logger := ctx.Value(loggerKey{})
if logger == nil {
return L.WithContext(ctx)
}
return logger.(*logrus.Entry)
}

149
vendor/github.com/containerd/containerd/log/context_deprecated.go generated vendored Normal file
View File

@@ -0,0 +1,149 @@
/*
Copyright The containerd Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package log
import (
"context"
"github.com/containerd/log"
)
// G is a shorthand for [GetLogger].
//
// Deprecated: use [log.G].
var G = log.G
// L is an alias for the standard logger.
//
// Deprecated: use [log.L].
var L = log.L
// Fields type to pass to "WithFields".
//
// Deprecated: use [log.Fields].
type Fields = log.Fields
// Entry is a logging entry.
//
// Deprecated: use [log.Entry].
type Entry = log.Entry
// RFC3339NanoFixed is [time.RFC3339Nano] with nanoseconds padded using
// zeros to ensure the formatted time is always the same number of
// characters.
//
// Deprecated: use [log.RFC3339NanoFixed].
const RFC3339NanoFixed = log.RFC3339NanoFixed
// Level is a logging level.
//
// Deprecated: use [log.Level].
type Level = log.Level
// Supported log levels.
const (
// TraceLevel level.
//
// Deprecated: use [log.TraceLevel].
TraceLevel Level = log.TraceLevel
// DebugLevel level.
//
// Deprecated: use [log.DebugLevel].
DebugLevel Level = log.DebugLevel
// InfoLevel level.
//
// Deprecated: use [log.InfoLevel].
InfoLevel Level = log.InfoLevel
// WarnLevel level.
//
// Deprecated: use [log.WarnLevel].
WarnLevel Level = log.WarnLevel
// ErrorLevel level
//
// Deprecated: use [log.ErrorLevel].
ErrorLevel Level = log.ErrorLevel
// FatalLevel level.
//
// Deprecated: use [log.FatalLevel].
FatalLevel Level = log.FatalLevel
// PanicLevel level.
//
// Deprecated: use [log.PanicLevel].
PanicLevel Level = log.PanicLevel
)
// SetLevel sets log level globally. It returns an error if the given
// level is not supported.
//
// Deprecated: use [log.SetLevel].
func SetLevel(level string) error {
return log.SetLevel(level)
}
// GetLevel returns the current log level.
//
// Deprecated: use [log.GetLevel].
func GetLevel() log.Level {
return log.GetLevel()
}
// OutputFormat specifies a log output format.
//
// Deprecated: use [log.OutputFormat].
type OutputFormat = log.OutputFormat
// Supported log output formats.
const (
// TextFormat represents the text logging format.
//
// Deprecated: use [log.TextFormat].
TextFormat log.OutputFormat = "text"
// JSONFormat represents the JSON logging format.
//
// Deprecated: use [log.JSONFormat].
JSONFormat log.OutputFormat = "json"
)
// SetFormat sets the log output format.
//
// Deprecated: use [log.SetFormat].
func SetFormat(format OutputFormat) error {
return log.SetFormat(format)
}
// WithLogger returns a new context with the provided logger. Use in
// combination with logger.WithField(s) for great effect.
//
// Deprecated: use [log.WithLogger].
func WithLogger(ctx context.Context, logger *log.Entry) context.Context {
return log.WithLogger(ctx, logger)
}
// GetLogger retrieves the current logger from the context. If no logger is
// available, the default logger is returned.
//
// Deprecated: use [log.GetLogger].
func GetLogger(ctx context.Context) *log.Entry {
return log.GetLogger(ctx)
}

48
vendor/github.com/containerd/containerd/pkg/randutil/randutil.go generated vendored Normal file
View File

@@ -0,0 +1,48 @@
/*
Copyright The containerd Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
// Package randutil provides utilities for [cyrpto/rand].
package randutil
import (
"crypto/rand"
"math"
"math/big"
)
// Int63n is similar to [math/rand.Int63n] but uses [crypto/rand.Reader] under the hood.
func Int63n(n int64) int64 {
b, err := rand.Int(rand.Reader, big.NewInt(n))
if err != nil {
panic(err)
}
return b.Int64()
}
// Int63 is similar to [math/rand.Int63] but uses [crypto/rand.Reader] under the hood.
func Int63() int64 {
return Int63n(math.MaxInt64)
}
// Intn is similar to [math/rand.Intn] but uses [crypto/rand.Reader] under the hood.
func Intn(n int) int {
return int(Int63n(int64(n)))
}
// Int is similar to [math/rand.Int] but uses [crypto/rand.Reader] under the hood.
func Int() int {
return int(Int63())
}

98
vendor/github.com/containerd/containerd/platforms/cpuinfo.go generated vendored
View File

@@ -17,14 +17,9 @@
package platforms
import (
"bufio"
"fmt"
"os"
"runtime"
"strings"
"sync"
"github.com/containerd/containerd/errdefs"
"github.com/containerd/containerd/log"
)
@@ -37,95 +32,12 @@ var cpuVariantOnce sync.Once
func cpuVariant() string {
cpuVariantOnce.Do(func() {
if isArmArch(runtime.GOARCH) {
cpuVariantValue = getCPUVariant()
var err error
cpuVariantValue, err = getCPUVariant()
if err != nil {
log.L.Errorf("Error getCPUVariant for OS %s: %v", runtime.GOOS, err)
}
}
})
return cpuVariantValue
}
// For Linux, the kernel has already detected the ABI, ISA and Features.
// So we don't need to access the ARM registers to detect platform information
// by ourselves. We can just parse these information from /proc/cpuinfo
func getCPUInfo(pattern string) (info string, err error) {
if !isLinuxOS(runtime.GOOS) {
return "", fmt.Errorf("getCPUInfo for OS %s: %w", runtime.GOOS, errdefs.ErrNotImplemented)
}
cpuinfo, err := os.Open("/proc/cpuinfo")
if err != nil {
return "", err
}
defer cpuinfo.Close()
// Start to Parse the Cpuinfo line by line. For SMP SoC, we parse
// the first core is enough.
scanner := bufio.NewScanner(cpuinfo)
for scanner.Scan() {
newline := scanner.Text()
list := strings.Split(newline, ":")
if len(list) > 1 && strings.EqualFold(strings.TrimSpace(list[0]), pattern) {
return strings.TrimSpace(list[1]), nil
}
}
// Check whether the scanner encountered errors
err = scanner.Err()
if err != nil {
return "", err
}
return "", fmt.Errorf("getCPUInfo for pattern: %s: %w", pattern, errdefs.ErrNotFound)
}
func getCPUVariant() string {
if runtime.GOOS == "windows" || runtime.GOOS == "darwin" {
// Windows/Darwin only supports v7 for ARM32 and v8 for ARM64 and so we can use
// runtime.GOARCH to determine the variants
var variant string
switch runtime.GOARCH {
case "arm64":
variant = "v8"
case "arm":
variant = "v7"
default:
variant = "unknown"
}
return variant
}
variant, err := getCPUInfo("Cpu architecture")
if err != nil {
log.L.WithError(err).Error("failure getting variant")
return ""
}
// handle edge case for Raspberry Pi ARMv6 devices (which due to a kernel quirk, report "CPU architecture: 7")
// https://www.raspberrypi.org/forums/viewtopic.php?t=12614
if runtime.GOARCH == "arm" && variant == "7" {
model, err := getCPUInfo("model name")
if err == nil && strings.HasPrefix(strings.ToLower(model), "armv6-compatible") {
variant = "6"
}
}
switch strings.ToLower(variant) {
case "8", "aarch64":
variant = "v8"
case "7", "7m", "?(12)", "?(13)", "?(14)", "?(15)", "?(16)", "?(17)":
variant = "v7"
case "6", "6tej":
variant = "v6"
case "5", "5t", "5te", "5tej":
variant = "v5"
case "4", "4t":
variant = "v4"
case "3":
variant = "v3"
default:
variant = "unknown"
}
return variant
}

161
vendor/github.com/containerd/containerd/platforms/cpuinfo_linux.go generated vendored Normal file
View File

@@ -0,0 +1,161 @@
/*
Copyright The containerd Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package platforms
import (
"bufio"
"bytes"
"fmt"
"os"
"runtime"
"strings"
"github.com/containerd/containerd/errdefs"
"golang.org/x/sys/unix"
)
// getMachineArch retrieves the machine architecture through system call
func getMachineArch() (string, error) {
var uname unix.Utsname
err := unix.Uname(&uname)
if err != nil {
return "", err
}
arch := string(uname.Machine[:bytes.IndexByte(uname.Machine[:], 0)])
return arch, nil
}
// For Linux, the kernel has already detected the ABI, ISA and Features.
// So we don't need to access the ARM registers to detect platform information
// by ourselves. We can just parse these information from /proc/cpuinfo
func getCPUInfo(pattern string) (info string, err error) {
cpuinfo, err := os.Open("/proc/cpuinfo")
if err != nil {
return "", err
}
defer cpuinfo.Close()
// Start to Parse the Cpuinfo line by line. For SMP SoC, we parse
// the first core is enough.
scanner := bufio.NewScanner(cpuinfo)
for scanner.Scan() {
newline := scanner.Text()
list := strings.Split(newline, ":")
if len(list) > 1 && strings.EqualFold(strings.TrimSpace(list[0]), pattern) {
return strings.TrimSpace(list[1]), nil
}
}
// Check whether the scanner encountered errors
err = scanner.Err()
if err != nil {
return "", err
}
return "", fmt.Errorf("getCPUInfo for pattern %s: %w", pattern, errdefs.ErrNotFound)
}
// getCPUVariantFromArch get CPU variant from arch through a system call
func getCPUVariantFromArch(arch string) (string, error) {
var variant string
arch = strings.ToLower(arch)
if arch == "aarch64" {
variant = "8"
} else if arch[0:4] == "armv" && len(arch) >= 5 {
//Valid arch format is in form of armvXx
switch arch[3:5] {
case "v8":
variant = "8"
case "v7":
variant = "7"
case "v6":
variant = "6"
case "v5":
variant = "5"
case "v4":
variant = "4"
case "v3":
variant = "3"
default:
variant = "unknown"
}
} else {
return "", fmt.Errorf("getCPUVariantFromArch invalid arch: %s, %w", arch, errdefs.ErrInvalidArgument)
}
return variant, nil
}
// getCPUVariant returns cpu variant for ARM
// We first try reading "Cpu architecture" field from /proc/cpuinfo
// If we can't find it, then fall back using a system call
// This is to cover running ARM in emulated environment on x86 host as this field in /proc/cpuinfo
// was not present.
func getCPUVariant() (string, error) {
variant, err := getCPUInfo("Cpu architecture")
if err != nil {
if errdefs.IsNotFound(err) {
//Let's try getting CPU variant from machine architecture
arch, err := getMachineArch()
if err != nil {
return "", fmt.Errorf("failure getting machine architecture: %v", err)
}
variant, err = getCPUVariantFromArch(arch)
if err != nil {
return "", fmt.Errorf("failure getting CPU variant from machine architecture: %v", err)
}
} else {
return "", fmt.Errorf("failure getting CPU variant: %v", err)
}
}
// handle edge case for Raspberry Pi ARMv6 devices (which due to a kernel quirk, report "CPU architecture: 7")
// https://www.raspberrypi.org/forums/viewtopic.php?t=12614
if runtime.GOARCH == "arm" && variant == "7" {
model, err := getCPUInfo("model name")
if err == nil && strings.HasPrefix(strings.ToLower(model), "armv6-compatible") {
variant = "6"
}
}
switch strings.ToLower(variant) {
case "8", "aarch64":
variant = "v8"
case "7", "7m", "?(12)", "?(13)", "?(14)", "?(15)", "?(16)", "?(17)":
variant = "v7"
case "6", "6tej":
variant = "v6"
case "5", "5t", "5te", "5tej":
variant = "v5"
case "4", "4t":
variant = "v4"
case "3":
variant = "v3"
default:
variant = "unknown"
}
return variant, nil
}

59
vendor/github.com/containerd/containerd/platforms/cpuinfo_other.go generated vendored Normal file
View File

@@ -0,0 +1,59 @@
//go:build !linux
/*
Copyright The containerd Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package platforms
import (
"fmt"
"runtime"
"github.com/containerd/containerd/errdefs"
)
func getCPUVariant() (string, error) {
var variant string
if runtime.GOOS == "windows" || runtime.GOOS == "darwin" {
// Windows/Darwin only supports v7 for ARM32 and v8 for ARM64 and so we can use
// runtime.GOARCH to determine the variants
switch runtime.GOARCH {
case "arm64":
variant = "v8"
case "arm":
variant = "v7"
default:
variant = "unknown"
}
} else if runtime.GOOS == "freebsd" {
// FreeBSD supports ARMv6 and ARMv7 as well as ARMv4 and ARMv5 (though deprecated)
// detecting those variants is currently unimplemented
switch runtime.GOARCH {
case "arm64":
variant = "v8"
default:
variant = "unknown"
}
} else {
return "", fmt.Errorf("getCPUVariant for OS %s: %v", runtime.GOOS, errdefs.ErrNotImplemented)
}
return variant, nil
}

7
vendor/github.com/containerd/containerd/platforms/database.go generated vendored
View File

@@ -21,13 +21,6 @@ import (
"strings"
)
// isLinuxOS returns true if the operating system is Linux.
//
// The OS value should be normalized before calling this function.
func isLinuxOS(os string) bool {
return os == "linux"
}
// These function are generated from https://golang.org/src/go/build/syslist.go.
//
// We use switch statements because they are slightly faster than map lookups

1
vendor/github.com/containerd/containerd/platforms/defaults_darwin.go generated vendored
View File

@@ -1,5 +1,4 @@
//go:build darwin
// +build darwin
/*
Copyright The containerd Authors.

43
vendor/github.com/containerd/containerd/platforms/defaults_freebsd.go generated vendored Normal file
View File

@@ -0,0 +1,43 @@
/*
Copyright The containerd Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package platforms
import (
"runtime"
specs "github.com/opencontainers/image-spec/specs-go/v1"
)
// DefaultSpec returns the current platform's default platform specification.
func DefaultSpec() specs.Platform {
return specs.Platform{
OS: runtime.GOOS,
Architecture: runtime.GOARCH,
// The Variant field will be empty if arch != ARM.
Variant: cpuVariant(),
}
}
// Default returns the default matcher for the platform.
func Default() MatchComparer {
return Ordered(DefaultSpec(), specs.Platform{
OS: "linux",
Architecture: runtime.GOARCH,
// The Variant field will be empty if arch != ARM.
Variant: cpuVariant(),
})
}

3
vendor/github.com/containerd/containerd/platforms/defaults_unix.go generated vendored
View File

@@ -1,5 +1,4 @@
//go:build !windows && !darwin
// +build !windows,!darwin
//go:build !windows && !darwin && !freebsd
/*
Copyright The containerd Authors.

60
vendor/github.com/containerd/containerd/platforms/defaults_windows.go generated vendored
View File

@@ -22,7 +22,7 @@ import (
"strconv"
"strings"
imagespec "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/Microsoft/hcsshim/osversion"
specs "github.com/opencontainers/image-spec/specs-go/v1"
"golang.org/x/sys/windows"
)
@@ -39,29 +39,52 @@ func DefaultSpec() specs.Platform {
}
}
type matchComparer struct {
defaults Matcher
type windowsmatcher struct {
specs.Platform
osVersionPrefix string
defaultMatcher Matcher
}
// Match matches platform with the same windows major, minor
// and build version.
func (m matchComparer) Match(p specs.Platform) bool {
match := m.defaults.Match(p)
func (m windowsmatcher) Match(p specs.Platform) bool {
match := m.defaultMatcher.Match(p)
if match && p.OS == "windows" {
if strings.HasPrefix(p.OSVersion, m.osVersionPrefix) {
if match && m.OS == "windows" {
// HPC containers do not have OS version filled
if p.OSVersion == "" {
return true
}
return p.OSVersion == ""
hostOsVersion := GetOsVersion(m.osVersionPrefix)
ctrOsVersion := GetOsVersion(p.OSVersion)
return osversion.CheckHostAndContainerCompat(hostOsVersion, ctrOsVersion)
}
return match
}
func GetOsVersion(osVersionPrefix string) osversion.OSVersion {
parts := strings.Split(osVersionPrefix, ".")
if len(parts) < 3 {
return osversion.OSVersion{}
}
majorVersion, _ := strconv.Atoi(parts[0])
minorVersion, _ := strconv.Atoi(parts[1])
buildNumber, _ := strconv.Atoi(parts[2])
return osversion.OSVersion{
MajorVersion: uint8(majorVersion),
MinorVersion: uint8(minorVersion),
Build: uint16(buildNumber),
}
return false
}
// Less sorts matched platforms in front of other platforms.
// For matched platforms, it puts platforms with larger revision
// number in front.
func (m matchComparer) Less(p1, p2 imagespec.Platform) bool {
func (m windowsmatcher) Less(p1, p2 specs.Platform) bool {
m1, m2 := m.Match(p1), m.Match(p2)
if m1 && m2 {
r1, r2 := revision(p1.OSVersion), revision(p2.OSVersion)
@@ -82,14 +105,15 @@ func revision(v string) int {
return r
}
func prefix(v string) string {
parts := strings.Split(v, ".")
if len(parts) < 4 {
return v
}
return strings.Join(parts[0:3], ".")
}
// Default returns the current platform's default platform specification.
func Default() MatchComparer {
major, minor, build := windows.RtlGetNtVersionNumbers()
return matchComparer{
defaults: Ordered(DefaultSpec(), specs.Platform{
OS: "linux",
Architecture: runtime.GOARCH,
}),
osVersionPrefix: fmt.Sprintf("%d.%d.%d", major, minor, build),
}
return Only(DefaultSpec())
}

23
vendor/github.com/containerd/containerd/platforms/platforms.go generated vendored
View File

@@ -114,14 +114,18 @@ import (
"strconv"
"strings"
"github.com/containerd/containerd/errdefs"
specs "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/containerd/containerd/errdefs"
)
var (
specifierRe = regexp.MustCompile(`^[A-Za-z0-9_-]+$`)
)
// Platform is a type alias for convenience, so there is no need to import image-spec package everywhere.
type Platform = specs.Platform
// Matcher matches platforms specifications, provided by an image or runtime.
type Matcher interface {
Match(platform specs.Platform) bool
@@ -136,9 +140,7 @@ type Matcher interface {
//
// Applications should opt to use `Match` over directly parsing specifiers.
func NewMatcher(platform specs.Platform) Matcher {
return &matcher{
Platform: Normalize(platform),
}
return newDefaultMatcher(platform)
}
type matcher struct {
@@ -194,6 +196,10 @@ func Parse(specifier string) (specs.Platform, error) {
p.Variant = cpuVariant()
}
if p.OS == "windows" {
p.OSVersion = GetWindowsOsVersion()
}
return p, nil
}
@@ -216,6 +222,10 @@ func Parse(specifier string) (specs.Platform, error) {
p.Variant = ""
}
if p.OS == "windows" {
p.OSVersion = GetWindowsOsVersion()
}
return p, nil
case 3:
// we have a fully specified variant, this is rare
@@ -225,6 +235,10 @@ func Parse(specifier string) (specs.Platform, error) {
p.Variant = "v8"
}
if p.OS == "windows" {
p.OSVersion = GetWindowsOsVersion()
}
return p, nil
}
@@ -257,5 +271,6 @@ func Format(platform specs.Platform) string {
func Normalize(platform specs.Platform) specs.Platform {
platform.OS = normalizeOS(platform.OS)
platform.Architecture, platform.Variant = normalizeArch(platform.Architecture, platform.Variant)
return platform
}

34
vendor/github.com/containerd/containerd/platforms/platforms_other.go generated vendored Normal file
View File

@@ -0,0 +1,34 @@
//go:build !windows
/*
Copyright The containerd Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package platforms
import (
specs "github.com/opencontainers/image-spec/specs-go/v1"
)
// NewMatcher returns the default Matcher for containerd
func newDefaultMatcher(platform specs.Platform) Matcher {
return &matcher{
Platform: Normalize(platform),
}
}
func GetWindowsOsVersion() string {
return ""
}

42
vendor/github.com/containerd/containerd/platforms/platforms_windows.go generated vendored Normal file
View File

@@ -0,0 +1,42 @@
/*
Copyright The containerd Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package platforms
import (
"fmt"
specs "github.com/opencontainers/image-spec/specs-go/v1"
"golang.org/x/sys/windows"
)
// NewMatcher returns a Windows matcher that will match on osVersionPrefix if
// the platform is Windows otherwise use the default matcher
func newDefaultMatcher(platform specs.Platform) Matcher {
prefix := prefix(platform.OSVersion)
return windowsmatcher{
Platform: platform,
osVersionPrefix: prefix,
defaultMatcher: &matcher{
Platform: Normalize(platform),
},
}
}
func GetWindowsOsVersion() string {
major, minor, build := windows.RtlGetNtVersionNumbers()
return fmt.Sprintf("%d.%d.%d", major, minor, build)
}

58
vendor/github.com/containerd/containerd/reference/docker/helpers.go generated vendored Normal file
View File

@@ -0,0 +1,58 @@
/*
Copyright The containerd Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package docker
import "path"
// IsNameOnly returns true if reference only contains a repo name.
func IsNameOnly(ref Named) bool {
if _, ok := ref.(NamedTagged); ok {
return false
}
if _, ok := ref.(Canonical); ok {
return false
}
return true
}
// FamiliarName returns the familiar name string
// for the given named, familiarizing if needed.
func FamiliarName(ref Named) string {
if nn, ok := ref.(normalizedNamed); ok {
return nn.Familiar().Name()
}
return ref.Name()
}
// FamiliarString returns the familiar string representation
// for the given reference, familiarizing if needed.
func FamiliarString(ref Reference) string {
if nn, ok := ref.(normalizedNamed); ok {
return nn.Familiar().String()
}
return ref.String()
}
// FamiliarMatch reports whether ref matches the specified pattern.
// See https://godoc.org/path#Match for supported patterns.
func FamiliarMatch(pattern string, ref Reference) (bool, error) {
matched, err := path.Match(pattern, FamiliarString(ref))
if namedRef, isNamed := ref.(Named); isNamed && !matched {
matched, _ = path.Match(pattern, FamiliarName(namedRef))
}
return matched, err
}

196
vendor/github.com/containerd/containerd/reference/docker/normalize.go generated vendored Normal file
View File

@@ -0,0 +1,196 @@
/*
Copyright The containerd Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package docker
import (
"fmt"
"strings"
"github.com/opencontainers/go-digest"
)
var (
legacyDefaultDomain = "index.docker.io"
defaultDomain = "docker.io"
officialRepoName = "library"
defaultTag = "latest"
)
// normalizedNamed represents a name which has been
// normalized and has a familiar form. A familiar name
// is what is used in Docker UI. An example normalized
// name is "docker.io/library/ubuntu" and corresponding
// familiar name of "ubuntu".
type normalizedNamed interface {
Named
Familiar() Named
}
// ParseNormalizedNamed parses a string into a named reference
// transforming a familiar name from Docker UI to a fully
// qualified reference. If the value may be an identifier
// use ParseAnyReference.
func ParseNormalizedNamed(s string) (Named, error) {
if ok := anchoredIdentifierRegexp.MatchString(s); ok {
return nil, fmt.Errorf("invalid repository name (%s), cannot specify 64-byte hexadecimal strings", s)
}
domain, remainder := splitDockerDomain(s)
var remoteName string
if tagSep := strings.IndexRune(remainder, ':'); tagSep > -1 {
remoteName = remainder[:tagSep]
} else {
remoteName = remainder
}
if strings.ToLower(remoteName) != remoteName {
return nil, fmt.Errorf("invalid reference format: repository name (%s) must be lowercase", remoteName)
}
ref, err := Parse(domain + "/" + remainder)
if err != nil {
return nil, err
}
named, isNamed := ref.(Named)
if !isNamed {
return nil, fmt.Errorf("reference %s has no name", ref.String())
}
return named, nil
}
// ParseDockerRef normalizes the image reference following the docker convention. This is added
// mainly for backward compatibility.
// The reference returned can only be either tagged or digested. For reference contains both tag
// and digest, the function returns digested reference, e.g. docker.io/library/busybox:latest@
// sha256:7cc4b5aefd1d0cadf8d97d4350462ba51c694ebca145b08d7d41b41acc8db5aa will be returned as
// docker.io/library/busybox@sha256:7cc4b5aefd1d0cadf8d97d4350462ba51c694ebca145b08d7d41b41acc8db5aa.
func ParseDockerRef(ref string) (Named, error) {
named, err := ParseNormalizedNamed(ref)
if err != nil {
return nil, err
}
if _, ok := named.(NamedTagged); ok {
if canonical, ok := named.(Canonical); ok {
// The reference is both tagged and digested, only
// return digested.
newNamed, err := WithName(canonical.Name())
if err != nil {
return nil, err
}
newCanonical, err := WithDigest(newNamed, canonical.Digest())
if err != nil {
return nil, err
}
return newCanonical, nil
}
}
return TagNameOnly(named), nil
}
// splitDockerDomain splits a repository name to domain and remotename string.
// If no valid domain is found, the default domain is used. Repository name
// needs to be already validated before.
func splitDockerDomain(name string) (domain, remainder string) {
i := strings.IndexRune(name, '/')
if i == -1 || (!strings.ContainsAny(name[:i], ".:") && name[:i] != "localhost" && strings.ToLower(name[:i]) == name[:i]) {
domain, remainder = defaultDomain, name
} else {
domain, remainder = name[:i], name[i+1:]
}
if domain == legacyDefaultDomain {
domain = defaultDomain
}
if domain == defaultDomain && !strings.ContainsRune(remainder, '/') {
remainder = officialRepoName + "/" + remainder
}
return
}
// familiarizeName returns a shortened version of the name familiar
// to the Docker UI. Familiar names have the default domain
// "docker.io" and "library/" repository prefix removed.
// For example, "docker.io/library/redis" will have the familiar
// name "redis" and "docker.io/dmcgowan/myapp" will be "dmcgowan/myapp".
// Returns a familiarized named only reference.
func familiarizeName(named namedRepository) repository {
repo := repository{
domain: named.Domain(),
path: named.Path(),
}
if repo.domain == defaultDomain {
repo.domain = ""
// Handle official repositories which have the pattern "library/<official repo name>"
if split := strings.Split(repo.path, "/"); len(split) == 2 && split[0] == officialRepoName {
repo.path = split[1]
}
}
return repo
}
func (r reference) Familiar() Named {
return reference{
namedRepository: familiarizeName(r.namedRepository),
tag: r.tag,
digest: r.digest,
}
}
func (r repository) Familiar() Named {
return familiarizeName(r)
}
func (t taggedReference) Familiar() Named {
return taggedReference{
namedRepository: familiarizeName(t.namedRepository),
tag: t.tag,
}
}
func (c canonicalReference) Familiar() Named {
return canonicalReference{
namedRepository: familiarizeName(c.namedRepository),
digest: c.digest,
}
}
// TagNameOnly adds the default tag "latest" to a reference if it only has
// a repo name.
func TagNameOnly(ref Named) Named {
if IsNameOnly(ref) {
namedTagged, err := WithTag(ref, defaultTag)
if err != nil {
// Default tag must be valid, to create a NamedTagged
// type with non-validated input the WithTag function
// should be used instead
panic(err)
}
return namedTagged
}
return ref
}
// ParseAnyReference parses a reference string as a possible identifier,
// full digest, or familiar name.
func ParseAnyReference(ref string) (Reference, error) {
if ok := anchoredIdentifierRegexp.MatchString(ref); ok {
return digestReference("sha256:" + ref), nil
}
if dgst, err := digest.Parse(ref); err == nil {
return digestReference(dgst), nil
}
return ParseNormalizedNamed(ref)
}

453
vendor/github.com/containerd/containerd/reference/docker/reference.go generated vendored Normal file
View File

@@ -0,0 +1,453 @@
/*
Copyright The containerd Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
// Package docker provides a general type to represent any way of referencing images within the registry.
// Its main purpose is to abstract tags and digests (content-addressable hash).
//
// Grammar
//
// reference := name [ ":" tag ] [ "@" digest ]
// name := [domain '/'] path-component ['/' path-component]*
// domain := host [':' port-number]
// host := domain-name | IPv4address | \[ IPv6address \] ; rfc3986 appendix-A
// domain-name := domain-component ['.' domain-component]*
// domain-component := /([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9-]*[a-zA-Z0-9])/
// port-number := /[0-9]+/
// path-component := alpha-numeric [separator alpha-numeric]*
// alpha-numeric := /[a-z0-9]+/
// separator := /[_.]|__|[-]*/
//
// tag := /[\w][\w.-]{0,127}/
//
// digest := digest-algorithm ":" digest-hex
// digest-algorithm := digest-algorithm-component [ digest-algorithm-separator digest-algorithm-component ]*
// digest-algorithm-separator := /[+.-_]/
// digest-algorithm-component := /[A-Za-z][A-Za-z0-9]*/
// digest-hex := /[0-9a-fA-F]{32,}/ ; At least 128 bit digest value
//
// identifier := /[a-f0-9]{64}/
// short-identifier := /[a-f0-9]{6,64}/
package docker
import (
"errors"
"fmt"
"strings"
"github.com/opencontainers/go-digest"
)
const (
// NameTotalLengthMax is the maximum total number of characters in a repository name.
NameTotalLengthMax = 255
)
var (
// ErrReferenceInvalidFormat represents an error while trying to parse a string as a reference.
ErrReferenceInvalidFormat = errors.New("invalid reference format")
// ErrTagInvalidFormat represents an error while trying to parse a string as a tag.
ErrTagInvalidFormat = errors.New("invalid tag format")
// ErrDigestInvalidFormat represents an error while trying to parse a string as a tag.
ErrDigestInvalidFormat = errors.New("invalid digest format")
// ErrNameContainsUppercase is returned for invalid repository names that contain uppercase characters.
ErrNameContainsUppercase = errors.New("repository name must be lowercase")
// ErrNameEmpty is returned for empty, invalid repository names.
ErrNameEmpty = errors.New("repository name must have at least one component")
// ErrNameTooLong is returned when a repository name is longer than NameTotalLengthMax.
ErrNameTooLong = fmt.Errorf("repository name must not be more than %v characters", NameTotalLengthMax)
// ErrNameNotCanonical is returned when a name is not canonical.
ErrNameNotCanonical = errors.New("repository name must be canonical")
)
// Reference is an opaque object reference identifier that may include
// modifiers such as a hostname, name, tag, and digest.
type Reference interface {
// String returns the full reference
String() string
}
// Field provides a wrapper type for resolving correct reference types when
// working with encoding.
type Field struct {
reference Reference
}
// AsField wraps a reference in a Field for encoding.
func AsField(reference Reference) Field {
return Field{reference}
}
// Reference unwraps the reference type from the field to
// return the Reference object. This object should be
// of the appropriate type to further check for different
// reference types.
func (f Field) Reference() Reference {
return f.reference
}
// MarshalText serializes the field to byte text which
// is the string of the reference.
func (f Field) MarshalText() (p []byte, err error) {
return []byte(f.reference.String()), nil
}
// UnmarshalText parses text bytes by invoking the
// reference parser to ensure the appropriately
// typed reference object is wrapped by field.
func (f *Field) UnmarshalText(p []byte) error {
r, err := Parse(string(p))
if err != nil {
return err
}
f.reference = r
return nil
}
// Named is an object with a full name
type Named interface {
Reference
Name() string
}
// Tagged is an object which has a tag
type Tagged interface {
Reference
Tag() string
}
// NamedTagged is an object including a name and tag.
type NamedTagged interface {
Named
Tag() string
}
// Digested is an object which has a digest
// in which it can be referenced by
type Digested interface {
Reference
Digest() digest.Digest
}
// Canonical reference is an object with a fully unique
// name including a name with domain and digest
type Canonical interface {
Named
Digest() digest.Digest
}
// namedRepository is a reference to a repository with a name.
// A namedRepository has both domain and path components.
type namedRepository interface {
Named
Domain() string
Path() string
}
// Domain returns the domain part of the Named reference
func Domain(named Named) string {
if r, ok := named.(namedRepository); ok {
return r.Domain()
}
domain, _ := splitDomain(named.Name())
return domain
}
// Path returns the name without the domain part of the Named reference
func Path(named Named) (name string) {
if r, ok := named.(namedRepository); ok {
return r.Path()
}
_, path := splitDomain(named.Name())
return path
}
func splitDomain(name string) (string, string) {
match := anchoredNameRegexp.FindStringSubmatch(name)
if len(match) != 3 {
return "", name
}
return match[1], match[2]
}
// SplitHostname splits a named reference into a
// hostname and name string. If no valid hostname is
// found, the hostname is empty and the full value
// is returned as name
// DEPRECATED: Use Domain or Path
func SplitHostname(named Named) (string, string) {
if r, ok := named.(namedRepository); ok {
return r.Domain(), r.Path()
}
return splitDomain(named.Name())
}
// Parse parses s and returns a syntactically valid Reference.
// If an error was encountered it is returned, along with a nil Reference.
// NOTE: Parse will not handle short digests.
func Parse(s string) (Reference, error) {
matches := ReferenceRegexp.FindStringSubmatch(s)
if matches == nil {
if s == "" {
return nil, ErrNameEmpty
}
if ReferenceRegexp.FindStringSubmatch(strings.ToLower(s)) != nil {
return nil, ErrNameContainsUppercase
}
return nil, ErrReferenceInvalidFormat
}
if len(matches[1]) > NameTotalLengthMax {
return nil, ErrNameTooLong
}
var repo repository
nameMatch := anchoredNameRegexp.FindStringSubmatch(matches[1])
if len(nameMatch) == 3 {
repo.domain = nameMatch[1]
repo.path = nameMatch[2]
} else {
repo.domain = ""
repo.path = matches[1]
}
ref := reference{
namedRepository: repo,
tag: matches[2],
}
if matches[3] != "" {
var err error
ref.digest, err = digest.Parse(matches[3])
if err != nil {
return nil, err
}
}
r := getBestReferenceType(ref)
if r == nil {
return nil, ErrNameEmpty
}
return r, nil
}
// ParseNamed parses s and returns a syntactically valid reference implementing
// the Named interface. The reference must have a name and be in the canonical
// form, otherwise an error is returned.
// If an error was encountered it is returned, along with a nil Reference.
// NOTE: ParseNamed will not handle short digests.
func ParseNamed(s string) (Named, error) {
named, err := ParseNormalizedNamed(s)
if err != nil {
return nil, err
}
if named.String() != s {
return nil, ErrNameNotCanonical
}
return named, nil
}
// WithName returns a named object representing the given string. If the input
// is invalid ErrReferenceInvalidFormat will be returned.
func WithName(name string) (Named, error) {
if len(name) > NameTotalLengthMax {
return nil, ErrNameTooLong
}
match := anchoredNameRegexp.FindStringSubmatch(name)
if match == nil || len(match) != 3 {
return nil, ErrReferenceInvalidFormat
}
return repository{
domain: match[1],
path: match[2],
}, nil
}
// WithTag combines the name from "name" and the tag from "tag" to form a
// reference incorporating both the name and the tag.
func WithTag(name Named, tag string) (NamedTagged, error) {
if !anchoredTagRegexp.MatchString(tag) {
return nil, ErrTagInvalidFormat
}
var repo repository
if r, ok := name.(namedRepository); ok {
repo.domain = r.Domain()
repo.path = r.Path()
} else {
repo.path = name.Name()
}
if canonical, ok := name.(Canonical); ok {
return reference{
namedRepository: repo,
tag: tag,
digest: canonical.Digest(),
}, nil
}
return taggedReference{
namedRepository: repo,
tag: tag,
}, nil
}
// WithDigest combines the name from "name" and the digest from "digest" to form
// a reference incorporating both the name and the digest.
func WithDigest(name Named, digest digest.Digest) (Canonical, error) {
if !anchoredDigestRegexp.MatchString(digest.String()) {
return nil, ErrDigestInvalidFormat
}
var repo repository
if r, ok := name.(namedRepository); ok {
repo.domain = r.Domain()
repo.path = r.Path()
} else {
repo.path = name.Name()
}
if tagged, ok := name.(Tagged); ok {
return reference{
namedRepository: repo,
tag: tagged.Tag(),
digest: digest,
}, nil
}
return canonicalReference{
namedRepository: repo,
digest: digest,
}, nil
}
// TrimNamed removes any tag or digest from the named reference.
func TrimNamed(ref Named) Named {
repo := repository{}
if r, ok := ref.(namedRepository); ok {
repo.domain, repo.path = r.Domain(), r.Path()
} else {
repo.domain, repo.path = splitDomain(ref.Name())
}
return repo
}
func getBestReferenceType(ref reference) Reference {
if ref.Name() == "" {
// Allow digest only references
if ref.digest != "" {
return digestReference(ref.digest)
}
return nil
}
if ref.tag == "" {
if ref.digest != "" {
return canonicalReference{
namedRepository: ref.namedRepository,
digest: ref.digest,
}
}
return ref.namedRepository
}
if ref.digest == "" {
return taggedReference{
namedRepository: ref.namedRepository,
tag: ref.tag,
}
}
return ref
}
type reference struct {
namedRepository
tag string
digest digest.Digest
}
func (r reference) String() string {
return r.Name() + ":" + r.tag + "@" + r.digest.String()
}
func (r reference) Tag() string {
return r.tag
}
func (r reference) Digest() digest.Digest {
return r.digest
}
type repository struct {
domain string
path string
}
func (r repository) String() string {
return r.Name()
}
func (r repository) Name() string {
if r.domain == "" {
return r.path
}
return r.domain + "/" + r.path
}
func (r repository) Domain() string {
return r.domain
}
func (r repository) Path() string {
return r.path
}
type digestReference digest.Digest
func (d digestReference) String() string {
return digest.Digest(d).String()
}
func (d digestReference) Digest() digest.Digest {
return digest.Digest(d)
}
type taggedReference struct {
namedRepository
tag string
}
func (t taggedReference) String() string {
return t.Name() + ":" + t.tag
}
func (t taggedReference) Tag() string {
return t.tag
}
type canonicalReference struct {
namedRepository
digest digest.Digest
}
func (c canonicalReference) String() string {
return c.Name() + "@" + c.digest.String()
}
func (c canonicalReference) Digest() digest.Digest {
return c.digest
}

191
vendor/github.com/containerd/containerd/reference/docker/regexp.go generated vendored Normal file
View File

@@ -0,0 +1,191 @@
/*
Copyright The containerd Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package docker
import "regexp"
var (
// alphaNumeric defines the alpha numeric atom, typically a
// component of names. This only allows lower case characters and digits.
alphaNumeric = `[a-z0-9]+`
// separator defines the separators allowed to be embedded in name
// components. This allow one period, one or two underscore and multiple
// dashes. Repeated dashes and underscores are intentionally treated
// differently. In order to support valid hostnames as name components,
// supporting repeated dash was added. Additionally double underscore is
// now allowed as a separator to loosen the restriction for previously
// supported names.
separator = `(?:[._]|__|[-]*)`
// nameComponent restricts registry path component names to start
// with at least one letter or number, with following parts able to be
// separated by one period, one or two underscore and multiple dashes.
nameComponent = expression(
alphaNumeric,
optional(repeated(separator, alphaNumeric)))
// domainNameComponent restricts the registry domain component of a
// repository name to start with a component as defined by DomainRegexp.
domainNameComponent = `(?:[a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9-]*[a-zA-Z0-9])`
// ipv6address are enclosed between square brackets and may be represented
// in many ways, see rfc5952. Only IPv6 in compressed or uncompressed format
// are allowed, IPv6 zone identifiers (rfc6874) or Special addresses such as
// IPv4-Mapped are deliberately excluded.
ipv6address = expression(
literal(`[`), `(?:[a-fA-F0-9:]+)`, literal(`]`),
)
// domainName defines the structure of potential domain components
// that may be part of image names. This is purposely a subset of what is
// allowed by DNS to ensure backwards compatibility with Docker image
// names. This includes IPv4 addresses on decimal format.
domainName = expression(
domainNameComponent,
optional(repeated(literal(`.`), domainNameComponent)),
)
// host defines the structure of potential domains based on the URI
// Host subcomponent on rfc3986. It may be a subset of DNS domain name,
// or an IPv4 address in decimal format, or an IPv6 address between square
// brackets (excluding zone identifiers as defined by rfc6874 or special
// addresses such as IPv4-Mapped).
host = `(?:` + domainName + `|` + ipv6address + `)`
// allowed by the URI Host subcomponent on rfc3986 to ensure backwards
// compatibility with Docker image names.
domain = expression(
host,
optional(literal(`:`), `[0-9]+`))
// DomainRegexp defines the structure of potential domain components
// that may be part of image names. This is purposely a subset of what is
// allowed by DNS to ensure backwards compatibility with Docker image
// names.
DomainRegexp = regexp.MustCompile(domain)
tag = `[\w][\w.-]{0,127}`
// TagRegexp matches valid tag names. From docker/docker:graph/tags.go.
TagRegexp = regexp.MustCompile(tag)
anchoredTag = anchored(tag)
// anchoredTagRegexp matches valid tag names, anchored at the start and
// end of the matched string.
anchoredTagRegexp = regexp.MustCompile(anchoredTag)
digestPat = `[A-Za-z][A-Za-z0-9]*(?:[-_+.][A-Za-z][A-Za-z0-9]*)*[:][[:xdigit:]]{32,}`
// DigestRegexp matches valid digests.
DigestRegexp = regexp.MustCompile(digestPat)
anchoredDigest = anchored(digestPat)
// anchoredDigestRegexp matches valid digests, anchored at the start and
// end of the matched string.
anchoredDigestRegexp = regexp.MustCompile(anchoredDigest)
namePat = expression(
optional(domain, literal(`/`)),
nameComponent,
optional(repeated(literal(`/`), nameComponent)))
// NameRegexp is the format for the name component of references. The
// regexp has capturing groups for the domain and name part omitting
// the separating forward slash from either.
NameRegexp = regexp.MustCompile(namePat)
anchoredName = anchored(
optional(capture(domain), literal(`/`)),
capture(nameComponent,
optional(repeated(literal(`/`), nameComponent))))
// anchoredNameRegexp is used to parse a name value, capturing the
// domain and trailing components.
anchoredNameRegexp = regexp.MustCompile(anchoredName)
referencePat = anchored(capture(namePat),
optional(literal(":"), capture(tag)),
optional(literal("@"), capture(digestPat)))
// ReferenceRegexp is the full supported format of a reference. The regexp
// is anchored and has capturing groups for name, tag, and digest
// components.
ReferenceRegexp = regexp.MustCompile(referencePat)
identifier = `([a-f0-9]{64})`
// IdentifierRegexp is the format for string identifier used as a
// content addressable identifier using sha256. These identifiers
// are like digests without the algorithm, since sha256 is used.
IdentifierRegexp = regexp.MustCompile(identifier)
shortIdentifier = `([a-f0-9]{6,64})`
// ShortIdentifierRegexp is the format used to represent a prefix
// of an identifier. A prefix may be used to match a sha256 identifier
// within a list of trusted identifiers.
ShortIdentifierRegexp = regexp.MustCompile(shortIdentifier)
anchoredIdentifier = anchored(identifier)
// anchoredIdentifierRegexp is used to check or match an
// identifier value, anchored at start and end of string.
anchoredIdentifierRegexp = regexp.MustCompile(anchoredIdentifier)
)
// literal compiles s into a literal regular expression, escaping any regexp
// reserved characters.
func literal(s string) string {
re := regexp.MustCompile(regexp.QuoteMeta(s))
if _, complete := re.LiteralPrefix(); !complete {
panic("must be a literal")
}
return re.String()
}
// expression defines a full expression, where each regular expression must
// follow the previous.
func expression(res ...string) string {
var s string
for _, re := range res {
s += re
}
return s
}
// optional wraps the expression in a non-capturing group and makes the
// production optional.
func optional(res ...string) string {
return group(expression(res...)) + `?`
}
// repeated wraps the regexp in a non-capturing group to get one or more
// matches.
func repeated(res ...string) string {
return group(expression(res...)) + `+`
}
// group wraps the regexp in a non-capturing group.
func group(res ...string) string {
return `(?:` + expression(res...) + `)`
}
// capture wraps the expression in a capturing group.
func capture(res ...string) string {
return `(` + expression(res...) + `)`
}
// anchored anchors the regular expression by adding start and end delimiters.
func anchored(res ...string) string {
return `^` + expression(res...) + `$`
}

73
vendor/github.com/containerd/containerd/reference/docker/sort.go generated vendored Normal file
View File

@@ -0,0 +1,73 @@
/*
Copyright The containerd Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package docker
import (
"sort"
)
// Sort sorts string references preferring higher information references
// The precedence is as follows:
// 1. Name + Tag + Digest
// 2. Name + Tag
// 3. Name + Digest
// 4. Name
// 5. Digest
// 6. Parse error
func Sort(references []string) []string {
var prefs []Reference
var bad []string
for _, ref := range references {
pref, err := ParseAnyReference(ref)
if err != nil {
bad = append(bad, ref)
} else {
prefs = append(prefs, pref)
}
}
sort.Slice(prefs, func(a, b int) bool {
ar := refRank(prefs[a])
br := refRank(prefs[b])
if ar == br {
return prefs[a].String() < prefs[b].String()
}
return ar < br
})
sort.Strings(bad)
var refs []string
for _, pref := range prefs {
refs = append(refs, pref.String())
}
return append(refs, bad...)
}
func refRank(ref Reference) uint8 {
if _, ok := ref.(Named); ok {
if _, ok = ref.(Tagged); ok {
if _, ok = ref.(Digested); ok {
return 1
}
return 2
}
if _, ok = ref.(Digested); ok {
return 3
}
return 4
}
return 5
}

9
vendor/github.com/containerd/containerd/remotes/docker/auth/fetch.go generated vendored
View File

@@ -29,7 +29,6 @@ import (
"github.com/containerd/containerd/log"
remoteserrors "github.com/containerd/containerd/remotes/errors"
"github.com/containerd/containerd/version"
"golang.org/x/net/context/ctxhttp"
)
var (
@@ -115,7 +114,7 @@ func FetchTokenWithOAuth(ctx context.Context, client *http.Client, headers http.
form.Set("access_type", "offline")
}
req, err := http.NewRequest("POST", to.Realm, strings.NewReader(form.Encode()))
req, err := http.NewRequestWithContext(ctx, http.MethodPost, to.Realm, strings.NewReader(form.Encode()))
if err != nil {
return nil, err
}
@@ -127,7 +126,7 @@ func FetchTokenWithOAuth(ctx context.Context, client *http.Client, headers http.
req.Header.Set("User-Agent", "containerd/"+version.Version)
}
resp, err := ctxhttp.Do(ctx, client, req)
resp, err := client.Do(req)
if err != nil {
return nil, err
}
@@ -162,7 +161,7 @@ type FetchTokenResponse struct {
// FetchToken fetches a token using a GET request
func FetchToken(ctx context.Context, client *http.Client, headers http.Header, to TokenOptions) (*FetchTokenResponse, error) {
req, err := http.NewRequest("GET", to.Realm, nil)
req, err := http.NewRequestWithContext(ctx, http.MethodGet, to.Realm, nil)
if err != nil {
return nil, err
}
@@ -194,7 +193,7 @@ func FetchToken(ctx context.Context, client *http.Client, headers http.Header, t
req.URL.RawQuery = reqParams.Encode()
resp, err := ctxhttp.Do(ctx, client, req)
resp, err := client.Do(req)
if err != nil {
return nil, err
}

26
vendor/github.com/containerd/containerd/remotes/docker/authorizer.go generated vendored
View File

@@ -29,7 +29,6 @@ import (
"github.com/containerd/containerd/log"
"github.com/containerd/containerd/remotes/docker/auth"
remoteerrors "github.com/containerd/containerd/remotes/errors"
"github.com/sirupsen/logrus"
)
type dockerAuthorizer struct {
@@ -45,13 +44,6 @@ type dockerAuthorizer struct {
onFetchRefreshToken OnFetchRefreshToken
}
// NewAuthorizer creates a Docker authorizer using the provided function to
// get credentials for the token server or basic auth.
// Deprecated: Use NewDockerAuthorizer
func NewAuthorizer(client *http.Client, f func(string) (string, string, error)) Authorizer {
return NewDockerAuthorizer(WithAuthClient(client), WithAuthCreds(f))
}
type authorizerConfig struct {
credentials func(string) (string, string, error)
client *http.Client
@@ -194,15 +186,15 @@ func (a *dockerAuthorizer) AddResponses(ctx context.Context, responses []*http.R
return err
}
if username != "" && secret != "" {
common := auth.TokenOptions{
Username: username,
Secret: secret,
}
a.handlers[host] = newAuthHandler(a.client, a.header, c.Scheme, common)
return nil
if username == "" || secret == "" {
return fmt.Errorf("%w: no basic auth credentials", ErrInvalidAuthorization)
}
a.handlers[host] = newAuthHandler(a.client, a.header, c.Scheme, auth.TokenOptions{
Username: username,
Secret: secret,
})
return nil
}
}
return fmt.Errorf("failed to find supported auth scheme: %w", errdefs.ErrNotImplemented)
@@ -319,7 +311,7 @@ func (ah *authHandler) doBearerAuth(ctx context.Context) (token, refreshToken st
}
return resp.Token, resp.RefreshToken, nil
}
log.G(ctx).WithFields(logrus.Fields{
log.G(ctx).WithFields(log.Fields{
"status": errStatus.Status,
"body": string(errStatus.Body),
}).Debugf("token request failed")

55
vendor/github.com/containerd/containerd/remotes/docker/converter_fuzz.go generated vendored Normal file
View File

@@ -0,0 +1,55 @@
//go:build gofuzz
/*
Copyright The containerd Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package docker
import (
"context"
"os"
fuzz "github.com/AdaLogics/go-fuzz-headers"
"github.com/containerd/containerd/content/local"
"github.com/containerd/containerd/log"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/sirupsen/logrus"
)
func FuzzConvertManifest(data []byte) int {
ctx := context.Background()
// Do not log the message below
// level=warning msg="do nothing for media type: ..."
log.G(ctx).Logger.SetLevel(logrus.PanicLevel)
f := fuzz.NewConsumer(data)
desc := ocispec.Descriptor{}
err := f.GenerateStruct(&desc)
if err != nil {
return 0
}
tmpdir, err := os.MkdirTemp("", "fuzzing-")
if err != nil {
return 0
}
cs, err := local.NewStore(tmpdir)
if err != nil {
return 0
}
_, _ = ConvertManifest(ctx, cs, desc)
return 1
}

108
vendor/github.com/containerd/containerd/remotes/docker/fetcher.go generated vendored
View File

@@ -29,6 +29,7 @@ import (
"github.com/containerd/containerd/errdefs"
"github.com/containerd/containerd/images"
"github.com/containerd/containerd/log"
digest "github.com/opencontainers/go-digest"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
)
@@ -52,18 +53,17 @@ func (r dockerFetcher) Fetch(ctx context.Context, desc ocispec.Descriptor) (io.R
return newHTTPReadSeeker(desc.Size, func(offset int64) (io.ReadCloser, error) {
// firstly try fetch via external urls
for _, us := range desc.URLs {
ctx = log.WithLogger(ctx, log.G(ctx).WithField("url", us))
u, err := url.Parse(us)
if err != nil {
log.G(ctx).WithError(err).Debug("failed to parse")
log.G(ctx).WithError(err).Debugf("failed to parse %q", us)
continue
}
if u.Scheme != "http" && u.Scheme != "https" {
log.G(ctx).Debug("non-http(s) alternative url is unsupported")
continue
}
log.G(ctx).Debug("trying alternative url")
ctx = log.WithLogger(ctx, log.G(ctx).WithField("url", u))
log.G(ctx).Info("request")
// Try this first, parse it
host := RegistryHost{
@@ -151,8 +151,106 @@ func (r dockerFetcher) Fetch(ctx context.Context, desc ocispec.Descriptor) (io.R
})
}
func (r dockerFetcher) createGetReq(ctx context.Context, host RegistryHost, ps ...string) (*request, int64, error) {
headReq := r.request(host, http.MethodHead, ps...)
if err := headReq.addNamespace(r.refspec.Hostname()); err != nil {
return nil, 0, err
}
headResp, err := headReq.doWithRetries(ctx, nil)
if err != nil {
return nil, 0, err
}
if headResp.Body != nil {
headResp.Body.Close()
}
if headResp.StatusCode > 299 {
return nil, 0, fmt.Errorf("unexpected HEAD status code %v: %s", headReq.String(), headResp.Status)
}
getReq := r.request(host, http.MethodGet, ps...)
if err := getReq.addNamespace(r.refspec.Hostname()); err != nil {
return nil, 0, err
}
return getReq, headResp.ContentLength, nil
}
func (r dockerFetcher) FetchByDigest(ctx context.Context, dgst digest.Digest) (io.ReadCloser, ocispec.Descriptor, error) {
var desc ocispec.Descriptor
ctx = log.WithLogger(ctx, log.G(ctx).WithField("digest", dgst))
hosts := r.filterHosts(HostCapabilityPull)
if len(hosts) == 0 {
return nil, desc, fmt.Errorf("no pull hosts: %w", errdefs.ErrNotFound)
}
ctx, err := ContextWithRepositoryScope(ctx, r.refspec, false)
if err != nil {
return nil, desc, err
}
var (
getReq *request
sz int64
firstErr error
)
for _, host := range r.hosts {
getReq, sz, err = r.createGetReq(ctx, host, "blobs", dgst.String())
if err == nil {
break
}
// Store the error for referencing later
if firstErr == nil {
firstErr = err
}
}
if getReq == nil {
// Fall back to the "manifests" endpoint
for _, host := range r.hosts {
getReq, sz, err = r.createGetReq(ctx, host, "manifests", dgst.String())
if err == nil {
break
}
// Store the error for referencing later
if firstErr == nil {
firstErr = err
}
}
}
if getReq == nil {
if errdefs.IsNotFound(firstErr) {
firstErr = fmt.Errorf("could not fetch content %v from remote: %w", dgst, errdefs.ErrNotFound)
}
if firstErr == nil {
firstErr = fmt.Errorf("could not fetch content %v from remote: (unknown)", dgst)
}
return nil, desc, firstErr
}
seeker, err := newHTTPReadSeeker(sz, func(offset int64) (io.ReadCloser, error) {
return r.open(ctx, getReq, "", offset)
})
if err != nil {
return nil, desc, err
}
desc = ocispec.Descriptor{
MediaType: "application/octet-stream",
Digest: dgst,
Size: sz,
}
return seeker, desc, nil
}
func (r dockerFetcher) open(ctx context.Context, req *request, mediatype string, offset int64) (_ io.ReadCloser, retErr error) {
req.header.Set("Accept", strings.Join([]string{mediatype, `*/*`}, ", "))
if mediatype == "" {
req.header.Set("Accept", "*/*")
} else {
req.header.Set("Accept", strings.Join([]string{mediatype, `*/*`}, ", "))
}
if offset > 0 {
// Note: "Accept-Ranges: bytes" cannot be trusted as some endpoints

81
vendor/github.com/containerd/containerd/remotes/docker/fetcher_fuzz.go generated vendored Normal file
View File

@@ -0,0 +1,81 @@
//go:build gofuzz
/*
Copyright The containerd Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package docker
import (
"context"
"fmt"
"io"
"net/http"
"net/http/httptest"
"net/url"
refDocker "github.com/containerd/containerd/reference/docker"
)
func FuzzFetcher(data []byte) int {
dataLen := len(data)
if dataLen == 0 {
return -1
}
s := httptest.NewServer(http.HandlerFunc(func(rw http.ResponseWriter, r *http.Request) {
rw.Header().Set("content-range", fmt.Sprintf("bytes %d-%d/%d", 0, dataLen-1, dataLen))
rw.Header().Set("content-length", fmt.Sprintf("%d", dataLen))
rw.Write(data)
}))
defer s.Close()
u, err := url.Parse(s.URL)
if err != nil {
return 0
}
f := dockerFetcher{&dockerBase{
repository: "nonempty",
}}
host := RegistryHost{
Client: s.Client(),
Host: u.Host,
Scheme: u.Scheme,
Path: u.Path,
}
ctx := context.Background()
req := f.request(host, http.MethodGet)
rc, err := f.open(ctx, req, "", 0)
if err != nil {
return 0
}
b, err := io.ReadAll(rc)
if err != nil {
return 0
}
expected := data
if len(b) != len(expected) {
panic("len of request is not equal to len of expected but should be")
}
return 1
}
func FuzzParseDockerRef(data []byte) int {
_, _ = refDocker.ParseDockerRef(string(data))
return 1
}

7
vendor/github.com/containerd/containerd/remotes/docker/handler.go generated vendored
View File

@@ -30,11 +30,6 @@ import (
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
)
var (
// labelDistributionSource describes the source blob comes from.
labelDistributionSource = "containerd.io/distribution.source"
)
// AppendDistributionSourceLabel updates the label of blob with distribution source.
func AppendDistributionSourceLabel(manager content.Manager, ref string) (images.HandlerFunc, error) {
refspec, err := reference.Parse(ref)
@@ -108,7 +103,7 @@ func appendDistributionSourceLabel(originLabel, repo string) string {
}
func distributionSourceLabelKey(source string) string {
return fmt.Sprintf("%s.%s", labelDistributionSource, source)
return fmt.Sprintf("%s.%s", labels.LabelDistributionSource, source)
}
// selectRepositoryMountCandidate will select the repo which has longest

10
vendor/github.com/containerd/containerd/remotes/docker/httpreadseeker.go generated vendored
View File

@@ -76,6 +76,16 @@ func (hrs *httpReadSeeker) Read(p []byte) (n int, err error) {
if _, err2 := hrs.reader(); err2 == nil {
return n, nil
}
} else if err == io.EOF {
// The CRI's imagePullProgressTimeout relies on responseBody.Close to
// update the process monitor's status. If the err is io.EOF, close
// the connection since there is no more available data.
if hrs.rc != nil {
if clsErr := hrs.rc.Close(); clsErr != nil {
log.L.WithError(clsErr).Error("httpReadSeeker: failed to close ReadCloser after io.EOF")
}
hrs.rc = nil
}
}
return
}

27
vendor/github.com/containerd/containerd/remotes/docker/pusher.go generated vendored
View File

@@ -23,6 +23,7 @@ import (
"io"
"net/http"
"net/url"
"path"
"strings"
"sync"
"time"
@@ -137,6 +138,9 @@ func (p dockerPusher) push(ctx context.Context, desc ocispec.Descriptor, ref str
if exists {
p.tracker.SetStatus(ref, Status{
Committed: true,
PushStatus: PushStatus{
Exists: true,
},
Status: content.Status{
Ref: ref,
Total: desc.Size,
@@ -164,6 +168,7 @@ func (p dockerPusher) push(ctx context.Context, desc ocispec.Descriptor, ref str
// Start upload request
req = p.request(host, http.MethodPost, "blobs", "uploads/")
mountedFrom := ""
var resp *http.Response
if fromRepo := selectRepositoryMountCandidate(p.refspec, desc.Annotations); fromRepo != "" {
preq := requestWithMountFrom(req, desc.Digest.String(), fromRepo)
@@ -180,17 +185,23 @@ func (p dockerPusher) push(ctx context.Context, desc ocispec.Descriptor, ref str
return nil, err
}
if resp.StatusCode == http.StatusUnauthorized {
switch resp.StatusCode {
case http.StatusUnauthorized:
log.G(ctx).Debugf("failed to mount from repository %s", fromRepo)
resp.Body.Close()
resp = nil
case http.StatusCreated:
mountedFrom = path.Join(p.refspec.Hostname(), fromRepo)
}
}
if resp == nil {
resp, err = req.doWithRetries(ctx, nil)
if err != nil {
if errors.Is(err, ErrInvalidAuthorization) {
return nil, fmt.Errorf("push access denied, repository does not exist or may require authorization: %w", err)
}
return nil, err
}
}
@@ -201,6 +212,9 @@ func (p dockerPusher) push(ctx context.Context, desc ocispec.Descriptor, ref str
case http.StatusCreated:
p.tracker.SetStatus(ref, Status{
Committed: true,
PushStatus: PushStatus{
MountedFrom: mountedFrom,
},
Status: content.Status{
Ref: ref,
Total: desc.Size,
@@ -235,13 +249,16 @@ func (p dockerPusher) push(ctx context.Context, desc ocispec.Descriptor, ref str
}
if lurl.Host != lhost.Host || lhost.Scheme != lurl.Scheme {
lhost.Scheme = lurl.Scheme
lhost.Host = lurl.Host
log.G(ctx).WithField("host", lhost.Host).WithField("scheme", lhost.Scheme).Debug("upload changed destination")
// Strip authorizer if change to host or scheme
lhost.Authorizer = nil
// Check if different than what was requested, accounting for fallback in the transport layer
requested := resp.Request.URL
if requested.Host != lhost.Host || requested.Scheme != lhost.Scheme {
// Strip authorizer if change to host or scheme
lhost.Authorizer = nil
log.G(ctx).WithField("host", lhost.Host).WithField("scheme", lhost.Scheme).Debug("upload changed destination, authorizer removed")
}
}
}
q := lurl.Query()

107
vendor/github.com/containerd/containerd/remotes/docker/resolver.go generated vendored
View File

@@ -18,6 +18,7 @@ package docker
import (
"context"
"crypto/tls"
"errors"
"fmt"
"io"
@@ -32,12 +33,12 @@ import (
"github.com/containerd/containerd/log"
"github.com/containerd/containerd/reference"
"github.com/containerd/containerd/remotes"
"github.com/containerd/containerd/remotes/docker/schema1"
"github.com/containerd/containerd/remotes/docker/schema1" //nolint:staticcheck // Ignore SA1019. Need to keep deprecated package for compatibility.
remoteerrors "github.com/containerd/containerd/remotes/errors"
"github.com/containerd/containerd/tracing"
"github.com/containerd/containerd/version"
digest "github.com/opencontainers/go-digest"
"github.com/opencontainers/go-digest"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/sirupsen/logrus"
"golang.org/x/net/context/ctxhttp"
)
var (
@@ -70,6 +71,9 @@ type Authorizer interface {
// unmodified. It may also add an `Authorization` header as
// "bearer <some bearer token>"
// "basic <base64 encoded credentials>"
//
// It may return remotes/errors.ErrUnexpectedStatus, which for example,
// can be used by the caller to find out the status code returned by the registry.
Authorize(context.Context, *http.Request) error
// AddResponses adds a 401 response for the authorizer to consider when
@@ -95,25 +99,30 @@ type ResolverOptions struct {
Tracker StatusTracker
// Authorizer is used to authorize registry requests
// Deprecated: use Hosts
//
// Deprecated: use Hosts.
Authorizer Authorizer
// Credentials provides username and secret given a host.
// If username is empty but a secret is given, that secret
// is interpreted as a long lived token.
// Deprecated: use Hosts
//
// Deprecated: use Hosts.
Credentials func(string) (string, string, error)
// Host provides the hostname given a namespace.
// Deprecated: use Hosts
//
// Deprecated: use Hosts.
Host func(string) (string, error)
// PlainHTTP specifies to use plain http and not https
// Deprecated: use Hosts
//
// Deprecated: use Hosts.
PlainHTTP bool
// Client is the http client to used when making registry requests
// Deprecated: use Hosts
//
// Deprecated: use Hosts.
Client *http.Client
}
@@ -140,6 +149,9 @@ func NewResolver(options ResolverOptions) remotes.Resolver {
if options.Headers == nil {
options.Headers = make(http.Header)
} else {
// make a copy of the headers to avoid race due to concurrent map write
options.Headers = options.Headers.Clone()
}
if _, ok := options.Headers["User-Agent"]; !ok {
options.Headers.Set("User-Agent", "containerd/"+version.Version)
@@ -152,7 +164,8 @@ func NewResolver(options ResolverOptions) remotes.Resolver {
images.MediaTypeDockerSchema2Manifest,
images.MediaTypeDockerSchema2ManifestList,
ocispec.MediaTypeImageManifest,
ocispec.MediaTypeImageIndex, "*/*"}, ", "))
ocispec.MediaTypeImageIndex, "*/*",
}, ", "))
} else {
resolveHeader["Accept"] = options.Headers["Accept"]
delete(options.Headers, "Accept")
@@ -299,11 +312,11 @@ func (r *dockerResolver) Resolve(ctx context.Context, ref string) (string, ocisp
if resp.StatusCode > 399 {
// Set firstErr when encountering the first non-404 status code.
if firstErr == nil {
firstErr = fmt.Errorf("pulling from host %s failed with status code %v: %v", host.Host, u, resp.Status)
firstErr = remoteerrors.NewUnexpectedStatusErr(resp)
}
continue // try another host
}
return "", ocispec.Descriptor{}, fmt.Errorf("pulling from host %s failed with unexpected status code %v: %v", host.Host, u, resp.Status)
return "", ocispec.Descriptor{}, remoteerrors.NewUnexpectedStatusErr(resp)
}
size := resp.ContentLength
contentType := getManifestMediaType(resp)
@@ -340,26 +353,31 @@ func (r *dockerResolver) Resolve(ctx context.Context, ref string) (string, ocisp
if err != nil {
return "", ocispec.Descriptor{}, err
}
defer resp.Body.Close()
bodyReader := countingReader{reader: resp.Body}
contentType = getManifestMediaType(resp)
if dgst == "" {
err = func() error {
defer resp.Body.Close()
if dgst != "" {
_, err = io.Copy(io.Discard, &bodyReader)
return err
}
if contentType == images.MediaTypeDockerSchema1Manifest {
b, err := schema1.ReadStripSignature(&bodyReader)
if err != nil {
return "", ocispec.Descriptor{}, err
return err
}
dgst = digest.FromBytes(b)
} else {
dgst, err = digest.FromReader(&bodyReader)
if err != nil {
return "", ocispec.Descriptor{}, err
}
return nil
}
} else if _, err := io.Copy(io.Discard, &bodyReader); err != nil {
dgst, err = digest.FromReader(&bodyReader)
return err
}()
if err != nil {
return "", ocispec.Descriptor{}, err
}
size = bodyReader.bytesRead
@@ -525,13 +543,14 @@ type request struct {
func (r *request) do(ctx context.Context) (*http.Response, error) {
u := r.host.Scheme + "://" + r.host.Host + r.path
req, err := http.NewRequest(r.method, u, nil)
req, err := http.NewRequestWithContext(ctx, r.method, u, nil)
if err != nil {
return nil, err
}
req.Header = http.Header{} // headers need to be copied to avoid concurrent map access
for k, v := range r.header {
req.Header[k] = v
if r.header == nil {
req.Header = http.Header{}
} else {
req.Header = r.header.Clone() // headers need to be copied to avoid concurrent map access
}
if r.body != nil {
body, err := r.body()
@@ -551,7 +570,7 @@ func (r *request) do(ctx context.Context) (*http.Response, error) {
return nil, fmt.Errorf("failed to authorize: %w", err)
}
var client = &http.Client{}
client := &http.Client{}
if r.host.Client != nil {
*client = *r.host.Client
}
@@ -567,7 +586,9 @@ func (r *request) do(ctx context.Context) (*http.Response, error) {
}
}
resp, err := ctxhttp.Do(ctx, client, req)
tracing.UpdateHTTPClient(client, tracing.Name("remotes.docker.resolver", "HTTPRequest"))
resp, err := client.Do(req)
if err != nil {
return nil, fmt.Errorf("failed to do request: %w", err)
}
@@ -630,7 +651,7 @@ func (r *request) String() string {
return r.host.Scheme + "://" + r.host.Host + r.path
}
func requestFields(req *http.Request) logrus.Fields {
func requestFields(req *http.Request) log.Fields {
fields := map[string]interface{}{
"request.method": req.Method,
}
@@ -648,10 +669,10 @@ func requestFields(req *http.Request) logrus.Fields {
}
}
return logrus.Fields(fields)
return fields
}
func responseFields(resp *http.Response) logrus.Fields {
func responseFields(resp *http.Response) log.Fields {
fields := map[string]interface{}{
"response.status": resp.Status,
}
@@ -666,7 +687,7 @@ func responseFields(resp *http.Response) logrus.Fields {
}
}
return logrus.Fields(fields)
return fields
}
// IsLocalhost checks if the registry host is local.
@@ -682,3 +703,27 @@ func IsLocalhost(host string) bool {
ip := net.ParseIP(host)
return ip.IsLoopback()
}
// HTTPFallback is an http.RoundTripper which allows fallback from https to http
// for registry endpoints with configurations for both http and TLS, such as
// defaulted localhost endpoints.
type HTTPFallback struct {
http.RoundTripper
}
func (f HTTPFallback) RoundTrip(r *http.Request) (*http.Response, error) {
resp, err := f.RoundTripper.RoundTrip(r)
var tlsErr tls.RecordHeaderError
if errors.As(err, &tlsErr) && string(tlsErr.RecordHeader[:]) == "HTTP/" {
// server gave HTTP response to HTTPS client
plainHTTPUrl := *r.URL
plainHTTPUrl.Scheme = "http"
plainHTTPRequest := *r
plainHTTPRequest.URL = &plainHTTPUrl
return f.RoundTripper.RoundTrip(&plainHTTPRequest)
}
return resp, err
}

12
vendor/github.com/containerd/containerd/remotes/docker/schema1/converter.go generated vendored
View File

@@ -14,6 +14,9 @@
limitations under the License.
*/
// Package schema1 provides a converter to fetch an image formatted in Docker Image Manifest v2, Schema 1.
//
// Deprecated: use images formatted in Docker Image Manifest v2, Schema 2, or OCI Image Spec v1.
package schema1
import (
@@ -33,6 +36,7 @@ import (
"github.com/containerd/containerd/content"
"github.com/containerd/containerd/errdefs"
"github.com/containerd/containerd/images"
"github.com/containerd/containerd/labels"
"github.com/containerd/containerd/log"
"github.com/containerd/containerd/remotes"
digest "github.com/opencontainers/go-digest"
@@ -363,12 +367,12 @@ func (c *Converter) fetchBlob(ctx context.Context, desc ocispec.Descriptor) erro
cinfo := content.Info{
Digest: desc.Digest,
Labels: map[string]string{
"containerd.io/uncompressed": state.diffID.String(),
labels.LabelUncompressed: state.diffID.String(),
labelDockerSchema1EmptyLayer: strconv.FormatBool(state.empty),
},
}
if _, err := c.contentStore.Update(ctx, cinfo, "labels.containerd.io/uncompressed", fmt.Sprintf("labels.%s", labelDockerSchema1EmptyLayer)); err != nil {
if _, err := c.contentStore.Update(ctx, cinfo, "labels."+labels.LabelUncompressed, fmt.Sprintf("labels.%s", labelDockerSchema1EmptyLayer)); err != nil {
return fmt.Errorf("failed to update uncompressed label: %w", err)
}
@@ -387,7 +391,7 @@ func (c *Converter) reuseLabelBlobState(ctx context.Context, desc ocispec.Descri
}
desc.Size = cinfo.Size
diffID, ok := cinfo.Labels["containerd.io/uncompressed"]
diffID, ok := cinfo.Labels[labels.LabelUncompressed]
if !ok {
return false, nil
}
@@ -406,7 +410,7 @@ func (c *Converter) reuseLabelBlobState(ctx context.Context, desc ocispec.Descri
bState := blobState{empty: isEmpty}
if bState.diffID, err = digest.Parse(diffID); err != nil {
log.G(ctx).WithField("id", desc.Digest).Warnf("failed to parse digest from label containerd.io/uncompressed: %v", diffID)
log.G(ctx).WithField("id", desc.Digest).Warnf("failed to parse digest from label %s: %v", labels.LabelUncompressed, diffID)
return false, nil
}

11
vendor/github.com/containerd/containerd/remotes/docker/status.go generated vendored
View File

@@ -36,6 +36,17 @@ type Status struct {
// UploadUUID is used by the Docker registry to reference blob uploads
UploadUUID string
// PushStatus contains status related to push.
PushStatus
}
type PushStatus struct {
// MountedFrom is the source content was cross-repo mounted from (empty if no cross-repo mount was performed).
MountedFrom string
// Exists indicates whether content already exists in the repository and wasn't uploaded.
Exists bool
}
// StatusTracker to track status of operations

2
vendor/github.com/containerd/containerd/remotes/errors/errors.go generated vendored
View File

@@ -33,7 +33,7 @@ type ErrUnexpectedStatus struct {
}
func (e ErrUnexpectedStatus) Error() string {
return fmt.Sprintf("unexpected status: %s", e.Status)
return fmt.Sprintf("unexpected status from %s request to %s: %s", e.RequestMethod, e.RequestURL, e.Status)
}
// NewUnexpectedStatusErr creates an ErrUnexpectedStatus from HTTP response

123
vendor/github.com/containerd/containerd/remotes/handlers.go generated vendored
View File

@@ -17,6 +17,7 @@
package remotes
import (
"bytes"
"context"
"errors"
"fmt"
@@ -27,10 +28,10 @@ import (
"github.com/containerd/containerd/content"
"github.com/containerd/containerd/errdefs"
"github.com/containerd/containerd/images"
"github.com/containerd/containerd/labels"
"github.com/containerd/containerd/log"
"github.com/containerd/containerd/platforms"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/sirupsen/logrus"
"golang.org/x/sync/semaphore"
)
@@ -90,7 +91,7 @@ func MakeRefKey(ctx context.Context, desc ocispec.Descriptor) string {
// recursive fetch.
func FetchHandler(ingester content.Ingester, fetcher Fetcher) images.HandlerFunc {
return func(ctx context.Context, desc ocispec.Descriptor) (subdescs []ocispec.Descriptor, err error) {
ctx = log.WithLogger(ctx, log.G(ctx).WithFields(logrus.Fields{
ctx = log.WithLogger(ctx, log.G(ctx).WithFields(log.Fields{
"digest": desc.Digest,
"mediatype": desc.MediaType,
"size": desc.Size,
@@ -100,20 +101,21 @@ func FetchHandler(ingester content.Ingester, fetcher Fetcher) images.HandlerFunc
case images.MediaTypeDockerSchema1Manifest:
return nil, fmt.Errorf("%v not supported", desc.MediaType)
default:
err := fetch(ctx, ingester, fetcher, desc)
err := Fetch(ctx, ingester, fetcher, desc)
if errdefs.IsAlreadyExists(err) {
return nil, nil
}
return nil, err
}
}
}
func fetch(ctx context.Context, ingester content.Ingester, fetcher Fetcher, desc ocispec.Descriptor) error {
// Fetch fetches the given digest into the provided ingester
func Fetch(ctx context.Context, ingester content.Ingester, fetcher Fetcher, desc ocispec.Descriptor) error {
log.G(ctx).Debug("fetch")
cw, err := content.OpenWriter(ctx, ingester, content.WithRef(MakeRefKey(ctx, desc)), content.WithDescriptor(desc))
if err != nil {
if errdefs.IsAlreadyExists(err) {
return nil
}
return err
}
defer cw.Close()
@@ -135,7 +137,11 @@ func fetch(ctx context.Context, ingester content.Ingester, fetcher Fetcher, desc
if err != nil && !errdefs.IsAlreadyExists(err) {
return fmt.Errorf("failed commit on ref %q: %w", ws.Ref, err)
}
return nil
return err
}
if desc.Size == int64(len(desc.Data)) {
return content.Copy(ctx, cw, bytes.NewReader(desc.Data), desc.Size, desc.Digest)
}
rc, err := fetcher.Fetch(ctx, desc)
@@ -151,7 +157,7 @@ func fetch(ctx context.Context, ingester content.Ingester, fetcher Fetcher, desc
// using a writer from the pusher.
func PushHandler(pusher Pusher, provider content.Provider) images.HandlerFunc {
return func(ctx context.Context, desc ocispec.Descriptor) ([]ocispec.Descriptor, error) {
ctx = log.WithLogger(ctx, log.G(ctx).WithFields(logrus.Fields{
ctx = log.WithLogger(ctx, log.G(ctx).WithFields(log.Fields{
"digest": desc.Digest,
"mediatype": desc.MediaType,
"size": desc.Size,
@@ -197,17 +203,26 @@ func push(ctx context.Context, provider content.Provider, pusher Pusher, desc oc
//
// Base handlers can be provided which will be called before any push specific
// handlers.
func PushContent(ctx context.Context, pusher Pusher, desc ocispec.Descriptor, store content.Store, limiter *semaphore.Weighted, platform platforms.MatchComparer, wrapper func(h images.Handler) images.Handler) error {
//
// If the passed in content.Provider is also a content.InfoProvider (such as
// content.Manager) then this will also annotate the distribution sources using
// labels prefixed with "containerd.io/distribution.source".
func PushContent(ctx context.Context, pusher Pusher, desc ocispec.Descriptor, store content.Provider, limiter *semaphore.Weighted, platform platforms.MatchComparer, wrapper func(h images.Handler) images.Handler) error {
var m sync.Mutex
manifestStack := []ocispec.Descriptor{}
manifests := []ocispec.Descriptor{}
indexStack := []ocispec.Descriptor{}
filterHandler := images.HandlerFunc(func(ctx context.Context, desc ocispec.Descriptor) ([]ocispec.Descriptor, error) {
switch desc.MediaType {
case images.MediaTypeDockerSchema2Manifest, ocispec.MediaTypeImageManifest,
images.MediaTypeDockerSchema2ManifestList, ocispec.MediaTypeImageIndex:
case images.MediaTypeDockerSchema2Manifest, ocispec.MediaTypeImageManifest:
m.Lock()
manifestStack = append(manifestStack, desc)
manifests = append(manifests, desc)
m.Unlock()
return nil, images.ErrStopHandler
case images.MediaTypeDockerSchema2ManifestList, ocispec.MediaTypeImageIndex:
m.Lock()
indexStack = append(indexStack, desc)
m.Unlock()
return nil, images.ErrStopHandler
default:
@@ -219,13 +234,14 @@ func PushContent(ctx context.Context, pusher Pusher, desc ocispec.Descriptor, st
platformFilterhandler := images.FilterPlatforms(images.ChildrenHandler(store), platform)
annotateHandler := annotateDistributionSourceHandler(platformFilterhandler, store)
var handler images.Handler
if m, ok := store.(content.InfoProvider); ok {
annotateHandler := annotateDistributionSourceHandler(platformFilterhandler, m)
handler = images.Handlers(annotateHandler, filterHandler, pushHandler)
} else {
handler = images.Handlers(platformFilterhandler, filterHandler, pushHandler)
}
var handler images.Handler = images.Handlers(
annotateHandler,
filterHandler,
pushHandler,
)
if wrapper != nil {
handler = wrapper(handler)
}
@@ -234,16 +250,18 @@ func PushContent(ctx context.Context, pusher Pusher, desc ocispec.Descriptor, st
return err
}
if err := images.Dispatch(ctx, pushHandler, limiter, manifests...); err != nil {
return err
}
// Iterate in reverse order as seen, parent always uploaded after child
for i := len(manifestStack) - 1; i >= 0; i-- {
_, err := pushHandler(ctx, manifestStack[i])
for i := len(indexStack) - 1; i >= 0; i-- {
err := images.Dispatch(ctx, pushHandler, limiter, indexStack[i])
if err != nil {
// TODO(estesp): until we have a more complete method for index push, we need to report
// missing dependencies in an index/manifest list by sensing the "400 Bad Request"
// as a marker for this problem
if (manifestStack[i].MediaType == ocispec.MediaTypeImageIndex ||
manifestStack[i].MediaType == images.MediaTypeDockerSchema2ManifestList) &&
errors.Unwrap(err) != nil && strings.Contains(errors.Unwrap(err).Error(), "400 Bad Request") {
if errors.Unwrap(err) != nil && strings.Contains(errors.Unwrap(err).Error(), "400 Bad Request") {
return fmt.Errorf("manifest list/index references to blobs and/or manifests are missing in your target registry: %w", err)
}
return err
@@ -327,14 +345,15 @@ func FilterManifestByPlatformHandler(f images.HandlerFunc, m platforms.Matcher)
// annotateDistributionSourceHandler add distribution source label into
// annotation of config or blob descriptor.
func annotateDistributionSourceHandler(f images.HandlerFunc, manager content.Manager) images.HandlerFunc {
func annotateDistributionSourceHandler(f images.HandlerFunc, provider content.InfoProvider) images.HandlerFunc {
return func(ctx context.Context, desc ocispec.Descriptor) ([]ocispec.Descriptor, error) {
children, err := f(ctx, desc)
if err != nil {
return nil, err
}
// only add distribution source for the config or blob data descriptor
// Distribution source is only used for config or blob but may be inherited from
// a manifest or manifest list
switch desc.MediaType {
case images.MediaTypeDockerSchema2Manifest, ocispec.MediaTypeImageManifest,
images.MediaTypeDockerSchema2ManifestList, ocispec.MediaTypeImageIndex:
@@ -342,27 +361,53 @@ func annotateDistributionSourceHandler(f images.HandlerFunc, manager content.Man
return children, nil
}
parentSourceAnnotations := desc.Annotations
var parentLabels map[string]string
if pi, err := provider.Info(ctx, desc.Digest); err != nil {
if !errdefs.IsNotFound(err) {
return nil, err
}
} else {
parentLabels = pi.Labels
}
for i := range children {
child := children[i]
info, err := manager.Info(ctx, child.Digest)
info, err := provider.Info(ctx, child.Digest)
if err != nil {
return nil, err
}
for k, v := range info.Labels {
if !strings.HasPrefix(k, "containerd.io/distribution.source.") {
continue
if !errdefs.IsNotFound(err) {
return nil, err
}
if child.Annotations == nil {
child.Annotations = map[string]string{}
}
child.Annotations[k] = v
}
copyDistributionSourceLabels(info.Labels, &child)
// Annotate with parent labels for cross repo mount or fetch.
// Parent sources may apply to all children since most registries
// enforce that children exist before the manifests.
copyDistributionSourceLabels(parentSourceAnnotations, &child)
copyDistributionSourceLabels(parentLabels, &child)
children[i] = child
}
return children, nil
}
}
func copyDistributionSourceLabels(from map[string]string, to *ocispec.Descriptor) {
for k, v := range from {
if !strings.HasPrefix(k, labels.LabelDistributionSource+".") {
continue
}
if to.Annotations == nil {
to.Annotations = make(map[string]string)
} else {
// Only propagate the parent label if the child doesn't already have it.
if _, has := to.Annotations[k]; has {
continue
}
}
to.Annotations[k] = v
}
}

16
vendor/github.com/containerd/containerd/remotes/resolver.go generated vendored
View File

@@ -21,6 +21,7 @@ import (
"io"
"github.com/containerd/containerd/content"
"github.com/opencontainers/go-digest"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
)
@@ -33,7 +34,7 @@ type Resolver interface {
// reference a specific host or be matched against a specific handler.
//
// The returned name should be used to identify the referenced entity.
// Dependending on the remote namespace, this may be immutable or mutable.
// Depending on the remote namespace, this may be immutable or mutable.
// While the name may differ from ref, it should itself be a valid ref.
//
// If the resolution fails, an error will be returned.
@@ -50,12 +51,23 @@ type Resolver interface {
Pusher(ctx context.Context, ref string) (Pusher, error)
}
// Fetcher fetches content
// Fetcher fetches content.
// A fetcher implementation may implement the FetcherByDigest interface too.
type Fetcher interface {
// Fetch the resource identified by the descriptor.
Fetch(ctx context.Context, desc ocispec.Descriptor) (io.ReadCloser, error)
}
// FetcherByDigest fetches content by the digest.
type FetcherByDigest interface {
// FetchByDigest fetches the resource identified by the digest.
//
// FetcherByDigest usually returns an incomplete descriptor.
// Typically, the media type is always set to "application/octet-stream",
// and the annotations are unset.
FetchByDigest(ctx context.Context, dgst digest.Digest) (io.ReadCloser, ocispec.Descriptor, error)
}
// Pusher pushes content
type Pusher interface {
// Push returns a content writer for the given resource identified

94
vendor/github.com/containerd/containerd/tracing/helpers.go generated vendored Normal file
View File

@@ -0,0 +1,94 @@
/*
Copyright The containerd Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package tracing
import (
"encoding/json"
"fmt"
"strings"
"go.opentelemetry.io/otel/attribute"
)
const (
spanDelimiter = "."
)
func makeSpanName(names ...string) string {
return strings.Join(names, spanDelimiter)
}
func any(k string, v interface{}) attribute.KeyValue {
if v == nil {
return attribute.String(k, "<nil>")
}
switch typed := v.(type) {
case bool:
return attribute.Bool(k, typed)
case []bool:
return attribute.BoolSlice(k, typed)
case int:
return attribute.Int(k, typed)
case []int:
return attribute.IntSlice(k, typed)
case int8:
return attribute.Int(k, int(typed))
case []int8:
ls := make([]int, 0, len(typed))
for _, i := range typed {
ls = append(ls, int(i))
}
return attribute.IntSlice(k, ls)
case int16:
return attribute.Int(k, int(typed))
case []int16:
ls := make([]int, 0, len(typed))
for _, i := range typed {
ls = append(ls, int(i))
}
return attribute.IntSlice(k, ls)
case int32:
return attribute.Int64(k, int64(typed))
case []int32:
ls := make([]int64, 0, len(typed))
for _, i := range typed {
ls = append(ls, int64(i))
}
return attribute.Int64Slice(k, ls)
case int64:
return attribute.Int64(k, typed)
case []int64:
return attribute.Int64Slice(k, typed)
case float64:
return attribute.Float64(k, typed)
case []float64:
return attribute.Float64Slice(k, typed)
case string:
return attribute.String(k, typed)
case []string:
return attribute.StringSlice(k, typed)
}
if stringer, ok := v.(fmt.Stringer); ok {
return attribute.String(k, stringer.String())
}
if b, err := json.Marshal(v); b != nil && err == nil {
return attribute.String(k, string(b))
}
return attribute.String(k, fmt.Sprintf("%v", v))
}

66
vendor/github.com/containerd/containerd/tracing/log.go generated vendored Normal file
View File

@@ -0,0 +1,66 @@
/*
Copyright The containerd Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package tracing
import (
"github.com/sirupsen/logrus"
"go.opentelemetry.io/otel/attribute"
"go.opentelemetry.io/otel/trace"
)
// NewLogrusHook creates a new logrus hook
func NewLogrusHook() *LogrusHook {
return &LogrusHook{}
}
// LogrusHook is a logrus hook which adds logrus events to active spans.
// If the span is not recording or the span context is invalid, the hook is a no-op.
type LogrusHook struct{}
// Levels returns the logrus levels that this hook is interested in.
func (h *LogrusHook) Levels() []logrus.Level {
return logrus.AllLevels
}
// Fire is called when a log event occurs.
func (h *LogrusHook) Fire(entry *logrus.Entry) error {
span := trace.SpanFromContext(entry.Context)
if span == nil {
return nil
}
if !span.SpanContext().IsValid() || !span.IsRecording() {
return nil
}
span.AddEvent(
entry.Message,
trace.WithAttributes(logrusDataToAttrs(entry.Data)...),
trace.WithAttributes(attribute.String("level", entry.Level.String())),
trace.WithTimestamp(entry.Time),
)
return nil
}
func logrusDataToAttrs(data logrus.Fields) []attribute.KeyValue {
attrs := make([]attribute.KeyValue, 0, len(data))
for k, v := range data {
attrs = append(attrs, any(k, v))
}
return attrs
}

129
vendor/github.com/containerd/containerd/tracing/tracing.go generated vendored Normal file
View File

@@ -0,0 +1,129 @@
/*
Copyright The containerd Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package tracing
import (
"context"
"net/http"
"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
"go.opentelemetry.io/otel"
"go.opentelemetry.io/otel/attribute"
"go.opentelemetry.io/otel/codes"
semconv "go.opentelemetry.io/otel/semconv/v1.21.0"
"go.opentelemetry.io/otel/trace"
)
// StartConfig defines configuration for a new span object.
type StartConfig struct {
spanOpts []trace.SpanStartOption
}
type SpanOpt func(config *StartConfig)
// WithHTTPRequest marks span as a HTTP request operation from client to server.
// It'll append attributes from the HTTP request object and mark it with `SpanKindClient` type.
//
// Deprecated: use upstream functionality from otelhttp directly instead. This function is kept for API compatibility
// but no longer works as expected due to required functionality no longer exported in OpenTelemetry libraries.
func WithHTTPRequest(_ *http.Request) SpanOpt {
return func(config *StartConfig) {
config.spanOpts = append(config.spanOpts,
trace.WithSpanKind(trace.SpanKindClient), // A client making a request to a server
)
}
}
// UpdateHTTPClient updates the http client with the necessary otel transport
func UpdateHTTPClient(client *http.Client, name string) {
client.Transport = otelhttp.NewTransport(
client.Transport,
otelhttp.WithSpanNameFormatter(func(operation string, r *http.Request) string {
return name
}),
)
}
// StartSpan starts child span in a context.
func StartSpan(ctx context.Context, opName string, opts ...SpanOpt) (context.Context, *Span) {
config := StartConfig{}
for _, fn := range opts {
fn(&config)
}
tracer := otel.Tracer("")
if parent := trace.SpanFromContext(ctx); parent != nil && parent.SpanContext().IsValid() {
tracer = parent.TracerProvider().Tracer("")
}
ctx, span := tracer.Start(ctx, opName, config.spanOpts...)
return ctx, &Span{otelSpan: span}
}
// SpanFromContext returns the current Span from the context.
func SpanFromContext(ctx context.Context) *Span {
return &Span{
otelSpan: trace.SpanFromContext(ctx),
}
}
// Span is wrapper around otel trace.Span.
// Span is the individual component of a trace. It represents a
// single named and timed operation of a workflow that is traced.
type Span struct {
otelSpan trace.Span
}
// End completes the span.
func (s *Span) End() {
s.otelSpan.End()
}
// AddEvent adds an event with provided name and options.
func (s *Span) AddEvent(name string, options ...trace.EventOption) {
s.otelSpan.AddEvent(name, options...)
}
// SetStatus sets the status of the current span.
// If an error is encountered, it records the error and sets span status to Error.
func (s *Span) SetStatus(err error) {
if err != nil {
s.otelSpan.RecordError(err)
s.otelSpan.SetStatus(codes.Error, err.Error())
} else {
s.otelSpan.SetStatus(codes.Ok, "")
}
}
// SetAttributes sets kv as attributes of the span.
func (s *Span) SetAttributes(kv ...attribute.KeyValue) {
s.otelSpan.SetAttributes(kv...)
}
// Name sets the span name by joining a list of strings in dot separated format.
func Name(names ...string) string {
return makeSpanName(names...)
}
// Attribute takes a key value pair and returns attribute.KeyValue type.
func Attribute(k string, v interface{}) attribute.KeyValue {
return any(k, v)
}
// HTTPStatusCodeAttributes generates attributes of the HTTP namespace as specified by the OpenTelemetry
// specification for a span.
func HTTPStatusCodeAttributes(code int) []attribute.KeyValue {
return []attribute.KeyValue{semconv.HTTPStatusCodeKey.Int(code)}
}

2
vendor/github.com/containerd/containerd/version/version.go generated vendored
View File

@@ -23,7 +23,7 @@ var (
Package = "github.com/containerd/containerd"
// Version holds the complete version number. Filled in at linking time.
Version = "1.6.18+unknown"
Version = "1.7.13+unknown"
// Revision is filled with the VCS (e.g. git) revision being used to build
// the program at linking time.

30
vendor/github.com/containerd/log/.golangci.yml generated vendored Normal file
View File

@@ -0,0 +1,30 @@
linters:
enable:
- exportloopref # Checks for pointers to enclosing loop variables
- gofmt
- goimports
- gosec
- ineffassign
- misspell
- nolintlint
- revive
- staticcheck
- tenv # Detects using os.Setenv instead of t.Setenv since Go 1.17
- unconvert
- unused
- vet
- dupword # Checks for duplicate words in the source code
disable:
- errcheck
run:
timeout: 5m
skip-dirs:
- api
- cluster
- design
- docs
- docs/man
- releases
- reports
- test # e2e scripts

191
vendor/github.com/containerd/log/LICENSE generated vendored Normal file
View File

@@ -0,0 +1,191 @@
Apache License
Version 2.0, January 2004
https://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
Copyright The containerd Authors
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
https://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

17
vendor/github.com/containerd/log/README.md generated vendored Normal file
View File

@@ -0,0 +1,17 @@
# log
A Go package providing a common logging interface across containerd repositories and a way for clients to use and configure logging in containerd packages.
This package is not intended to be used as a standalone logging package outside of the containerd ecosystem and is intended as an interface wrapper around a logging implementation.
In the future this package may be replaced with a common go logging interface.
## Project details
**log** is a containerd sub-project, licensed under the [Apache 2.0 license](./LICENSE).
As a containerd sub-project, you will find the:
* [Project governance](https://github.com/containerd/project/blob/main/GOVERNANCE.md),
* [Maintainers](https://github.com/containerd/project/blob/main/MAINTAINERS),
* and [Contributing guidelines](https://github.com/containerd/project/blob/main/CONTRIBUTING.md)
information in our [`containerd/project`](https://github.com/containerd/project) repository.

182
vendor/github.com/containerd/log/context.go generated vendored Normal file
View File

@@ -0,0 +1,182 @@
/*
Copyright The containerd Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
// Package log provides types and functions related to logging, passing
// loggers through a context, and attaching context to the logger.
//
// # Transitional types
//
// This package contains various types that are aliases for types in [logrus].
// These aliases are intended for transitioning away from hard-coding logrus
// as logging implementation. Consumers of this package are encouraged to use
// the type-aliases from this package instead of directly using their logrus
// equivalent.
//
// The intent is to replace these aliases with locally defined types and
// interfaces once all consumers are no longer directly importing logrus
// types.
//
// IMPORTANT: due to the transitional purpose of this package, it is not
// guaranteed for the full logrus API to be provided in the future. As
// outlined, these aliases are provided as a step to transition away from
// a specific implementation which, as a result, exposes the full logrus API.
// While no decisions have been made on the ultimate design and interface
// provided by this package, we do not expect carrying "less common" features.
package log
import (
"context"
"fmt"
"github.com/sirupsen/logrus"
)
// G is a shorthand for [GetLogger].
//
// We may want to define this locally to a package to get package tagged log
// messages.
var G = GetLogger
// L is an alias for the standard logger.
var L = &Entry{
Logger: logrus.StandardLogger(),
// Default is three fields plus a little extra room.
Data: make(Fields, 6),
}
type loggerKey struct{}
// Fields type to pass to "WithFields".
type Fields = map[string]any
// Entry is a logging entry. It contains all the fields passed with
// [Entry.WithFields]. It's finally logged when Trace, Debug, Info, Warn,
// Error, Fatal or Panic is called on it. These objects can be reused and
// passed around as much as you wish to avoid field duplication.
//
// Entry is a transitional type, and currently an alias for [logrus.Entry].
type Entry = logrus.Entry
// RFC3339NanoFixed is [time.RFC3339Nano] with nanoseconds padded using
// zeros to ensure the formatted time is always the same number of
// characters.
const RFC3339NanoFixed = "2006-01-02T15:04:05.000000000Z07:00"
// Level is a logging level.
type Level = logrus.Level
// Supported log levels.
const (
// TraceLevel level. Designates finer-grained informational events
// than [DebugLevel].
TraceLevel Level = logrus.TraceLevel
// DebugLevel level. Usually only enabled when debugging. Very verbose
// logging.
DebugLevel Level = logrus.DebugLevel
// InfoLevel level. General operational entries about what's going on
// inside the application.
InfoLevel Level = logrus.InfoLevel
// WarnLevel level. Non-critical entries that deserve eyes.
WarnLevel Level = logrus.WarnLevel
// ErrorLevel level. Logs errors that should definitely be noted.
// Commonly used for hooks to send errors to an error tracking service.
ErrorLevel Level = logrus.ErrorLevel
// FatalLevel level. Logs and then calls "logger.Exit(1)". It exits
// even if the logging level is set to Panic.
FatalLevel Level = logrus.FatalLevel
// PanicLevel level. This is the highest level of severity. Logs and
// then calls panic with the message passed to Debug, Info, ...
PanicLevel Level = logrus.PanicLevel
)
// SetLevel sets log level globally. It returns an error if the given
// level is not supported.
//
// level can be one of:
//
// - "trace" ([TraceLevel])
// - "debug" ([DebugLevel])
// - "info" ([InfoLevel])
// - "warn" ([WarnLevel])
// - "error" ([ErrorLevel])
// - "fatal" ([FatalLevel])
// - "panic" ([PanicLevel])
func SetLevel(level string) error {
lvl, err := logrus.ParseLevel(level)
if err != nil {
return err
}
L.Logger.SetLevel(lvl)
return nil
}
// GetLevel returns the current log level.
func GetLevel() Level {
return L.Logger.GetLevel()
}
// OutputFormat specifies a log output format.
type OutputFormat string
// Supported log output formats.
const (
// TextFormat represents the text logging format.
TextFormat OutputFormat = "text"
// JSONFormat represents the JSON logging format.
JSONFormat OutputFormat = "json"
)
// SetFormat sets the log output format ([TextFormat] or [JSONFormat]).
func SetFormat(format OutputFormat) error {
switch format {
case TextFormat:
L.Logger.SetFormatter(&logrus.TextFormatter{
TimestampFormat: RFC3339NanoFixed,
FullTimestamp: true,
})
return nil
case JSONFormat:
L.Logger.SetFormatter(&logrus.JSONFormatter{
TimestampFormat: RFC3339NanoFixed,
})
return nil
default:
return fmt.Errorf("unknown log format: %s", format)
}
}
// WithLogger returns a new context with the provided logger. Use in
// combination with logger.WithField(s) for great effect.
func WithLogger(ctx context.Context, logger *Entry) context.Context {
return context.WithValue(ctx, loggerKey{}, logger.WithContext(ctx))
}
// GetLogger retrieves the current logger from the context. If no logger is
// available, the default logger is returned.
func GetLogger(ctx context.Context) *Entry {
if logger := ctx.Value(loggerKey{}); logger != nil {
return logger.(*Entry)
}
return L.WithContext(ctx)
}

52
vendor/github.com/coreos/go-systemd/v22/journal/journal_unix.go generated vendored
View File

@@ -69,6 +69,58 @@ func Enabled() bool {
return true
}
// StderrIsJournalStream returns whether the process stderr is connected
// to the Journal's stream transport.
//
// This can be used for automatic protocol upgrading described in [Journal Native Protocol].
//
// Returns true if JOURNAL_STREAM environment variable is present,
// and stderr's device and inode numbers match it.
//
// Error is returned if unexpected error occurs: e.g. if JOURNAL_STREAM environment variable
// is present, but malformed, fstat syscall fails, etc.
//
// [Journal Native Protocol]: https://systemd.io/JOURNAL_NATIVE_PROTOCOL/#automatic-protocol-upgrading
func StderrIsJournalStream() (bool, error) {
return fdIsJournalStream(syscall.Stderr)
}
// StdoutIsJournalStream returns whether the process stdout is connected
// to the Journal's stream transport.
//
// Returns true if JOURNAL_STREAM environment variable is present,
// and stdout's device and inode numbers match it.
//
// Error is returned if unexpected error occurs: e.g. if JOURNAL_STREAM environment variable
// is present, but malformed, fstat syscall fails, etc.
//
// Most users should probably use [StderrIsJournalStream].
func StdoutIsJournalStream() (bool, error) {
return fdIsJournalStream(syscall.Stdout)
}
func fdIsJournalStream(fd int) (bool, error) {
journalStream := os.Getenv("JOURNAL_STREAM")
if journalStream == "" {
return false, nil
}
var expectedStat syscall.Stat_t
_, err := fmt.Sscanf(journalStream, "%d:%d", &expectedStat.Dev, &expectedStat.Ino)
if err != nil {
return false, fmt.Errorf("failed to parse JOURNAL_STREAM=%q: %v", journalStream, err)
}
var stat syscall.Stat_t
err = syscall.Fstat(fd, &stat)
if err != nil {
return false, err
}
match := stat.Dev == expectedStat.Dev && stat.Ino == expectedStat.Ino
return match, nil
}
// Send a message to the local systemd journal. vars is a map of journald
// fields to values. Fields must be composed of uppercase letters, numbers,
// and underscores, but must not start with an underscore. Within these

8
vendor/github.com/coreos/go-systemd/v22/journal/journal_windows.go generated vendored
View File

@@ -33,3 +33,11 @@ func Enabled() bool {
func Send(message string, priority Priority, vars map[string]string) error {
return errors.New("could not initialize socket to journald")
}
func StderrIsJournalStream() (bool, error) {
return false, nil
}
func StdoutIsJournalStream() (bool, error) {
return false, nil
}

4
vendor/github.com/grpc-ecosystem/grpc-gateway/v2/internal/httprule/fuzz.go generated vendored
View File

@@ -1,10 +1,10 @@
//go:build gofuzz
// +build gofuzz
package httprule
func Fuzz(data []byte) int {
_, err := Parse(string(data))
if err != nil {
if _, err := Parse(string(data)); err != nil {
return 0
}
return 0

19
vendor/github.com/grpc-ecosystem/grpc-gateway/v2/internal/httprule/parse.go generated vendored
View File

@@ -1,6 +1,7 @@
package httprule
import (
"errors"
"fmt"
"strings"
)
@@ -164,9 +165,9 @@ func (p *parser) segment() (segment, error) {
v, err := p.variable()
if err != nil {
return nil, fmt.Errorf("segment neither wildcards, literal or variable: %v", err)
return nil, fmt.Errorf("segment neither wildcards, literal or variable: %w", err)
}
return v, err
return v, nil
}
func (p *parser) literal() (segment, error) {
@@ -191,7 +192,7 @@ func (p *parser) variable() (segment, error) {
if _, err := p.accept("="); err == nil {
segs, err = p.segments()
if err != nil {
return nil, fmt.Errorf("invalid segment in variable %q: %v", path, err)
return nil, fmt.Errorf("invalid segment in variable %q: %w", path, err)
}
} else {
segs = []segment{wildcard{}}
@@ -213,12 +214,12 @@ func (p *parser) fieldPath() (string, error) {
}
components := []string{c}
for {
if _, err = p.accept("."); err != nil {
if _, err := p.accept("."); err != nil {
return strings.Join(components, "."), nil
}
c, err := p.accept(typeIdent)
if err != nil {
return "", fmt.Errorf("invalid field path component: %v", err)
return "", fmt.Errorf("invalid field path component: %w", err)
}
components = append(components, c)
}
@@ -237,10 +238,8 @@ const (
typeEOF = termType("$")
)
const (
// eof is the terminal symbol which always appears at the end of token sequence.
eof = "\u0000"
)
// eof is the terminal symbol which always appears at the end of token sequence.
const eof = "\u0000"
// accept tries to accept a token in "p".
// This function consumes a token and returns it if it matches to the specified "term".
@@ -334,7 +333,7 @@ func expectPChars(t string) error {
// expectIdent determines if "ident" is a valid identifier in .proto schema ([[:alpha:]_][[:alphanum:]_]*).
func expectIdent(ident string) error {
if ident == "" {
return fmt.Errorf("empty identifier")
return errors.New("empty identifier")
}
for pos, r := range ident {
switch {

4
vendor/github.com/grpc-ecosystem/grpc-gateway/v2/runtime/BUILD.bazel generated vendored
View File

@@ -27,7 +27,6 @@ go_library(
"//internal/httprule",
"//utilities",
"@go_googleapis//google/api:httpbody_go_proto",
"@io_bazel_rules_go//proto/wkt:field_mask_go_proto",
"@org_golang_google_grpc//codes",
"@org_golang_google_grpc//grpclog",
"@org_golang_google_grpc//health/grpc_health_v1",
@@ -38,6 +37,7 @@ go_library(
"@org_golang_google_protobuf//reflect/protoreflect",
"@org_golang_google_protobuf//reflect/protoregistry",
"@org_golang_google_protobuf//types/known/durationpb",
"@org_golang_google_protobuf//types/known/fieldmaskpb",
"@org_golang_google_protobuf//types/known/structpb",
"@org_golang_google_protobuf//types/known/timestamppb",
"@org_golang_google_protobuf//types/known/wrapperspb",
@@ -73,7 +73,6 @@ go_test(
"@go_googleapis//google/api:httpbody_go_proto",
"@go_googleapis//google/rpc:errdetails_go_proto",
"@go_googleapis//google/rpc:status_go_proto",
"@io_bazel_rules_go//proto/wkt:field_mask_go_proto",
"@org_golang_google_grpc//:go_default_library",
"@org_golang_google_grpc//codes",
"@org_golang_google_grpc//health/grpc_health_v1",
@@ -84,6 +83,7 @@ go_test(
"@org_golang_google_protobuf//testing/protocmp",
"@org_golang_google_protobuf//types/known/durationpb",
"@org_golang_google_protobuf//types/known/emptypb",
"@org_golang_google_protobuf//types/known/fieldmaskpb",
"@org_golang_google_protobuf//types/known/structpb",
"@org_golang_google_protobuf//types/known/timestamppb",
"@org_golang_google_protobuf//types/known/wrapperspb",

53
vendor/github.com/grpc-ecosystem/grpc-gateway/v2/runtime/context.go generated vendored
View File

@@ -13,6 +13,7 @@ import (
"time"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/grpclog"
"google.golang.org/grpc/metadata"
"google.golang.org/grpc/status"
)
@@ -35,11 +36,9 @@ const metadataHeaderBinarySuffix = "-Bin"
const xForwardedFor = "X-Forwarded-For"
const xForwardedHost = "X-Forwarded-Host"
var (
// DefaultContextTimeout is used for gRPC call context.WithTimeout whenever a Grpc-Timeout inbound
// header isn't present. If the value is 0 the sent `context` will not have a timeout.
DefaultContextTimeout = 0 * time.Second
)
// DefaultContextTimeout is used for gRPC call context.WithTimeout whenever a Grpc-Timeout inbound
// header isn't present. If the value is 0 the sent `context` will not have a timeout.
var DefaultContextTimeout = 0 * time.Second
// malformedHTTPHeaders lists the headers that the gRPC server may reject outright as malformed.
// See https://github.com/grpc/grpc-go/pull/4803#issuecomment-986093310 for more context.
@@ -101,12 +100,43 @@ func AnnotateIncomingContext(ctx context.Context, mux *ServeMux, req *http.Reque
return metadata.NewIncomingContext(ctx, md), nil
}
func isValidGRPCMetadataKey(key string) bool {
// Must be a valid gRPC "Header-Name" as defined here:
// https://github.com/grpc/grpc/blob/4b05dc88b724214d0c725c8e7442cbc7a61b1374/doc/PROTOCOL-HTTP2.md
// This means 0-9 a-z _ - .
// Only lowercase letters are valid in the wire protocol, but the client library will normalize
// uppercase ASCII to lowercase, so uppercase ASCII is also acceptable.
bytes := []byte(key) // gRPC validates strings on the byte level, not Unicode.
for _, ch := range bytes {
validLowercaseLetter := ch >= 'a' && ch <= 'z'
validUppercaseLetter := ch >= 'A' && ch <= 'Z'
validDigit := ch >= '0' && ch <= '9'
validOther := ch == '.' || ch == '-' || ch == '_'
if !validLowercaseLetter && !validUppercaseLetter && !validDigit && !validOther {
return false
}
}
return true
}
func isValidGRPCMetadataTextValue(textValue string) bool {
// Must be a valid gRPC "ASCII-Value" as defined here:
// https://github.com/grpc/grpc/blob/4b05dc88b724214d0c725c8e7442cbc7a61b1374/doc/PROTOCOL-HTTP2.md
// This means printable ASCII (including/plus spaces); 0x20 to 0x7E inclusive.
bytes := []byte(textValue) // gRPC validates strings on the byte level, not Unicode.
for _, ch := range bytes {
if ch < 0x20 || ch > 0x7E {
return false
}
}
return true
}
func annotateContext(ctx context.Context, mux *ServeMux, req *http.Request, rpcMethodName string, options ...AnnotateContextOption) (context.Context, metadata.MD, error) {
ctx = withRPCMethod(ctx, rpcMethodName)
for _, o := range options {
ctx = o(ctx)
}
var pairs []string
timeout := DefaultContextTimeout
if tm := req.Header.Get(metadataGrpcTimeout); tm != "" {
var err error
@@ -115,7 +145,7 @@ func annotateContext(ctx context.Context, mux *ServeMux, req *http.Request, rpcM
return nil, nil, status.Errorf(codes.InvalidArgument, "invalid grpc-timeout: %s", tm)
}
}
var pairs []string
for key, vals := range req.Header {
key = textproto.CanonicalMIMEHeaderKey(key)
for _, val := range vals {
@@ -124,6 +154,10 @@ func annotateContext(ctx context.Context, mux *ServeMux, req *http.Request, rpcM
pairs = append(pairs, "authorization", val)
}
if h, ok := mux.incomingHeaderMatcher(key); ok {
if !isValidGRPCMetadataKey(h) {
grpclog.Errorf("HTTP header name %q is not valid as gRPC metadata key; skipping", h)
continue
}
// Handles "-bin" metadata in grpc, since grpc will do another base64
// encode before sending to server, we need to decode it first.
if strings.HasSuffix(key, metadataHeaderBinarySuffix) {
@@ -133,6 +167,9 @@ func annotateContext(ctx context.Context, mux *ServeMux, req *http.Request, rpcM
}
val = string(b)
} else if !isValidGRPCMetadataTextValue(val) {
grpclog.Errorf("Value of HTTP header %q contains non-ASCII value (not valid as gRPC metadata): skipping", h)
continue
}
pairs = append(pairs, h, val)
}
@@ -281,8 +318,8 @@ func timeoutUnitToDuration(u uint8) (d time.Duration, ok bool) {
case 'n':
return time.Nanosecond, true
default:
return
}
return
}
// isPermanentHTTPHeader checks whether hdr belongs to the list of

46
vendor/github.com/grpc-ecosystem/grpc-gateway/v2/runtime/convert.go generated vendored
View File

@@ -37,7 +37,7 @@ func BoolSlice(val, sep string) ([]bool, error) {
for i, v := range s {
value, err := Bool(v)
if err != nil {
return values, err
return nil, err
}
values[i] = value
}
@@ -57,7 +57,7 @@ func Float64Slice(val, sep string) ([]float64, error) {
for i, v := range s {
value, err := Float64(v)
if err != nil {
return values, err
return nil, err
}
values[i] = value
}
@@ -81,7 +81,7 @@ func Float32Slice(val, sep string) ([]float32, error) {
for i, v := range s {
value, err := Float32(v)
if err != nil {
return values, err
return nil, err
}
values[i] = value
}
@@ -101,7 +101,7 @@ func Int64Slice(val, sep string) ([]int64, error) {
for i, v := range s {
value, err := Int64(v)
if err != nil {
return values, err
return nil, err
}
values[i] = value
}
@@ -125,7 +125,7 @@ func Int32Slice(val, sep string) ([]int32, error) {
for i, v := range s {
value, err := Int32(v)
if err != nil {
return values, err
return nil, err
}
values[i] = value
}
@@ -145,7 +145,7 @@ func Uint64Slice(val, sep string) ([]uint64, error) {
for i, v := range s {
value, err := Uint64(v)
if err != nil {
return values, err
return nil, err
}
values[i] = value
}
@@ -169,7 +169,7 @@ func Uint32Slice(val, sep string) ([]uint32, error) {
for i, v := range s {
value, err := Uint32(v)
if err != nil {
return values, err
return nil, err
}
values[i] = value
}
@@ -197,7 +197,7 @@ func BytesSlice(val, sep string) ([][]byte, error) {
for i, v := range s {
value, err := Bytes(v)
if err != nil {
return values, err
return nil, err
}
values[i] = value
}
@@ -209,8 +209,7 @@ func Timestamp(val string) (*timestamppb.Timestamp, error) {
var r timestamppb.Timestamp
val = strconv.Quote(strings.Trim(val, `"`))
unmarshaler := &protojson.UnmarshalOptions{}
err := unmarshaler.Unmarshal([]byte(val), &r)
if err != nil {
if err := unmarshaler.Unmarshal([]byte(val), &r); err != nil {
return nil, err
}
return &r, nil
@@ -221,8 +220,7 @@ func Duration(val string) (*durationpb.Duration, error) {
var r durationpb.Duration
val = strconv.Quote(strings.Trim(val, `"`))
unmarshaler := &protojson.UnmarshalOptions{}
err := unmarshaler.Unmarshal([]byte(val), &r)
if err != nil {
if err := unmarshaler.Unmarshal([]byte(val), &r); err != nil {
return nil, err
}
return &r, nil
@@ -257,66 +255,64 @@ func EnumSlice(val, sep string, enumValMap map[string]int32) ([]int32, error) {
for i, v := range s {
value, err := Enum(v, enumValMap)
if err != nil {
return values, err
return nil, err
}
values[i] = value
}
return values, nil
}
/*
Support for google.protobuf.wrappers on top of primitive types
*/
// Support for google.protobuf.wrappers on top of primitive types
// StringValue well-known type support as wrapper around string type
func StringValue(val string) (*wrapperspb.StringValue, error) {
return &wrapperspb.StringValue{Value: val}, nil
return wrapperspb.String(val), nil
}
// FloatValue well-known type support as wrapper around float32 type
func FloatValue(val string) (*wrapperspb.FloatValue, error) {
parsedVal, err := Float32(val)
return &wrapperspb.FloatValue{Value: parsedVal}, err
return wrapperspb.Float(parsedVal), err
}
// DoubleValue well-known type support as wrapper around float64 type
func DoubleValue(val string) (*wrapperspb.DoubleValue, error) {
parsedVal, err := Float64(val)
return &wrapperspb.DoubleValue{Value: parsedVal}, err
return wrapperspb.Double(parsedVal), err
}
// BoolValue well-known type support as wrapper around bool type
func BoolValue(val string) (*wrapperspb.BoolValue, error) {
parsedVal, err := Bool(val)
return &wrapperspb.BoolValue{Value: parsedVal}, err
return wrapperspb.Bool(parsedVal), err
}
// Int32Value well-known type support as wrapper around int32 type
func Int32Value(val string) (*wrapperspb.Int32Value, error) {
parsedVal, err := Int32(val)
return &wrapperspb.Int32Value{Value: parsedVal}, err
return wrapperspb.Int32(parsedVal), err
}
// UInt32Value well-known type support as wrapper around uint32 type
func UInt32Value(val string) (*wrapperspb.UInt32Value, error) {
parsedVal, err := Uint32(val)
return &wrapperspb.UInt32Value{Value: parsedVal}, err
return wrapperspb.UInt32(parsedVal), err
}
// Int64Value well-known type support as wrapper around int64 type
func Int64Value(val string) (*wrapperspb.Int64Value, error) {
parsedVal, err := Int64(val)
return &wrapperspb.Int64Value{Value: parsedVal}, err
return wrapperspb.Int64(parsedVal), err
}
// UInt64Value well-known type support as wrapper around uint64 type
func UInt64Value(val string) (*wrapperspb.UInt64Value, error) {
parsedVal, err := Uint64(val)
return &wrapperspb.UInt64Value{Value: parsedVal}, err
return wrapperspb.UInt64(parsedVal), err
}
// BytesValue well-known type support as wrapper around bytes[] type
func BytesValue(val string) (*wrapperspb.BytesValue, error) {
parsedVal, err := Bytes(val)
return &wrapperspb.BytesValue{Value: parsedVal}, err
return wrapperspb.Bytes(parsedVal), err
}

8
vendor/github.com/grpc-ecosystem/grpc-gateway/v2/runtime/errors.go generated vendored
View File

@@ -38,7 +38,7 @@ func HTTPStatusFromCode(code codes.Code) int {
case codes.OK:
return http.StatusOK
case codes.Canceled:
return http.StatusRequestTimeout
return 499
case codes.Unknown:
return http.StatusInternalServerError
case codes.InvalidArgument:
@@ -70,10 +70,10 @@ func HTTPStatusFromCode(code codes.Code) int {
return http.StatusServiceUnavailable
case codes.DataLoss:
return http.StatusInternalServerError
default:
grpclog.Infof("Unknown gRPC error code: %v", code)
return http.StatusInternalServerError
}
grpclog.Infof("Unknown gRPC error code: %v", code)
return http.StatusInternalServerError
}
// HTTPError uses the mux-configured error handler.

7
vendor/github.com/grpc-ecosystem/grpc-gateway/v2/runtime/fieldmask.go generated vendored
View File

@@ -2,13 +2,14 @@ package runtime
import (
"encoding/json"
"errors"
"fmt"
"io"
"sort"
"google.golang.org/genproto/protobuf/field_mask"
"google.golang.org/protobuf/proto"
"google.golang.org/protobuf/reflect/protoreflect"
field_mask "google.golang.org/protobuf/types/known/fieldmaskpb"
)
func getFieldByName(fields protoreflect.FieldDescriptors, name string) protoreflect.FieldDescriptor {
@@ -44,7 +45,7 @@ func FieldMaskFromRequestBody(r io.Reader, msg proto.Message) (*field_mask.Field
// if the item is an object, then enqueue all of its children
for k, v := range m {
if item.msg == nil {
return nil, fmt.Errorf("JSON structure did not match request type")
return nil, errors.New("JSON structure did not match request type")
}
fd := getFieldByName(item.msg.Descriptor().Fields(), k)
@@ -63,7 +64,7 @@ func FieldMaskFromRequestBody(r io.Reader, msg proto.Message) (*field_mask.Field
continue
}
if isProtobufAnyMessage(fd.Message()) {
if isProtobufAnyMessage(fd.Message()) && !fd.IsList() {
_, hasTypeField := v.(map[string]interface{})["@type"]
if hasTypeField {
queue = append(queue, fieldMaskPathItem{path: k})

16
vendor/github.com/grpc-ecosystem/grpc-gateway/v2/runtime/handler.go generated vendored
View File

@@ -85,12 +85,12 @@ func ForwardResponseStream(ctx context.Context, mux *ServeMux, marshaler Marshal
handleForwardResponseStreamError(ctx, wroteHeader, marshaler, w, req, mux, err, delimiter)
return
}
if _, err = w.Write(buf); err != nil {
if _, err := w.Write(buf); err != nil {
grpclog.Infof("Failed to send response chunk: %v", err)
return
}
wroteHeader = true
if _, err = w.Write(delimiter); err != nil {
if _, err := w.Write(delimiter); err != nil {
grpclog.Infof("Failed to send delimiter chunk: %v", err)
return
}
@@ -207,16 +207,16 @@ func handleForwardResponseStreamError(ctx context.Context, wroteHeader bool, mar
w.Header().Set("Content-Type", marshaler.ContentType(msg))
w.WriteHeader(HTTPStatusFromCode(st.Code()))
}
buf, merr := marshaler.Marshal(msg)
if merr != nil {
grpclog.Infof("Failed to marshal an error: %v", merr)
buf, err := marshaler.Marshal(msg)
if err != nil {
grpclog.Infof("Failed to marshal an error: %v", err)
return
}
if _, werr := w.Write(buf); werr != nil {
grpclog.Infof("Failed to notify error to client: %v", werr)
if _, err := w.Write(buf); err != nil {
grpclog.Infof("Failed to notify error to client: %v", err)
return
}
if _, derr := w.Write(delimiter); derr != nil {
if _, err := w.Write(delimiter); err != nil {
grpclog.Infof("Failed to send delimiter chunk: %v", err)
return
}

27
vendor/github.com/grpc-ecosystem/grpc-gateway/v2/runtime/marshal_jsonpb.go generated vendored
View File

@@ -92,23 +92,20 @@ func (j *JSONPb) marshalNonProtoField(v interface{}) ([]byte, error) {
if rv.Type().Elem().Implements(protoMessageType) {
var buf bytes.Buffer
err := buf.WriteByte('[')
if err != nil {
if err := buf.WriteByte('['); err != nil {
return nil, err
}
for i := 0; i < rv.Len(); i++ {
if i != 0 {
err = buf.WriteByte(',')
if err != nil {
if err := buf.WriteByte(','); err != nil {
return nil, err
}
}
if err = j.marshalTo(&buf, rv.Index(i).Interface().(proto.Message)); err != nil {
if err := j.marshalTo(&buf, rv.Index(i).Interface().(proto.Message)); err != nil {
return nil, err
}
}
err = buf.WriteByte(']')
if err != nil {
if err := buf.WriteByte(']'); err != nil {
return nil, err
}
@@ -117,17 +114,16 @@ func (j *JSONPb) marshalNonProtoField(v interface{}) ([]byte, error) {
if rv.Type().Elem().Implements(typeProtoEnum) {
var buf bytes.Buffer
err := buf.WriteByte('[')
if err != nil {
if err := buf.WriteByte('['); err != nil {
return nil, err
}
for i := 0; i < rv.Len(); i++ {
if i != 0 {
err = buf.WriteByte(',')
if err != nil {
if err := buf.WriteByte(','); err != nil {
return nil, err
}
}
var err error
if j.UseEnumNumbers {
_, err = buf.WriteString(strconv.FormatInt(rv.Index(i).Int(), 10))
} else {
@@ -137,8 +133,7 @@ func (j *JSONPb) marshalNonProtoField(v interface{}) ([]byte, error) {
return nil, err
}
}
err = buf.WriteByte(']')
if err != nil {
if err := buf.WriteByte(']'); err != nil {
return nil, err
}
@@ -219,8 +214,7 @@ func decodeJSONPb(d *json.Decoder, unmarshaler protojson.UnmarshalOptions, v int
// Decode into bytes for marshalling
var b json.RawMessage
err := d.Decode(&b)
if err != nil {
if err := d.Decode(&b); err != nil {
return err
}
@@ -239,8 +233,7 @@ func decodeNonProtoField(d *json.Decoder, unmarshaler protojson.UnmarshalOptions
if rv.Type().ConvertibleTo(typeProtoMessage) {
// Decode into bytes for marshalling
var b json.RawMessage
err := d.Decode(&b)
if err != nil {
if err := d.Decode(&b); err != nil {
return err
}

9
vendor/github.com/grpc-ecosystem/grpc-gateway/v2/runtime/marshal_proto.go generated vendored
View File

@@ -1,10 +1,8 @@
package runtime
import (
"io"
"errors"
"io/ioutil"
"io"
"google.golang.org/protobuf/proto"
)
@@ -38,7 +36,7 @@ func (*ProtoMarshaller) Unmarshal(data []byte, value interface{}) error {
// NewDecoder returns a Decoder which reads proto stream from "reader".
func (marshaller *ProtoMarshaller) NewDecoder(reader io.Reader) Decoder {
return DecoderFunc(func(value interface{}) error {
buffer, err := ioutil.ReadAll(reader)
buffer, err := io.ReadAll(reader)
if err != nil {
return err
}
@@ -53,8 +51,7 @@ func (marshaller *ProtoMarshaller) NewEncoder(writer io.Writer) Encoder {
if err != nil {
return err
}
_, err = writer.Write(buffer)
if err != nil {
if _, err := writer.Write(buffer); err != nil {
return err
}

92
vendor/github.com/grpc-ecosystem/grpc-gateway/v2/runtime/mux.go generated vendored
View File

@@ -43,9 +43,7 @@ const (
UnescapingModeDefault = UnescapingModeLegacy
)
var (
encodedPathSplitter = regexp.MustCompile("(/|%2F)")
)
var encodedPathSplitter = regexp.MustCompile("(/|%2F)")
// A HandlerFunc handles a specific pair of path pattern and HTTP method.
type HandlerFunc func(w http.ResponseWriter, r *http.Request, pathParams map[string]string)
@@ -82,7 +80,7 @@ func WithForwardResponseOption(forwardResponseOption func(context.Context, http.
}
}
// WithEscapingType sets the escaping type. See the definitions of UnescapingMode
// WithUnescapingMode sets the escaping type. See the definitions of UnescapingMode
// for more information.
func WithUnescapingMode(mode UnescapingMode) ServeMuxOption {
return func(serveMux *ServeMux) {
@@ -103,13 +101,14 @@ func SetQueryParameterParser(queryParameterParser QueryParameterParser) ServeMux
type HeaderMatcherFunc func(string) (string, bool)
// DefaultHeaderMatcher is used to pass http request headers to/from gRPC context. This adds permanent HTTP header
// keys (as specified by the IANA) to gRPC context with grpcgateway- prefix. HTTP headers that start with
// 'Grpc-Metadata-' are mapped to gRPC metadata after removing prefix 'Grpc-Metadata-'.
// keys (as specified by the IANA, e.g: Accept, Cookie, Host) to the gRPC metadata with the grpcgateway- prefix. If you want to know which headers are considered permanent, you can view the isPermanentHTTPHeader function.
// HTTP headers that start with 'Grpc-Metadata-' are mapped to gRPC metadata after removing the prefix 'Grpc-Metadata-'.
// Other headers are not added to the gRPC metadata.
func DefaultHeaderMatcher(key string) (string, bool) {
key = textproto.CanonicalMIMEHeaderKey(key)
if isPermanentHTTPHeader(key) {
switch key = textproto.CanonicalMIMEHeaderKey(key); {
case isPermanentHTTPHeader(key):
return MetadataPrefix + key, true
} else if strings.HasPrefix(key, MetadataHeaderPrefix) {
case strings.HasPrefix(key, MetadataHeaderPrefix):
return key[len(MetadataHeaderPrefix):], true
}
return "", false
@@ -232,7 +231,6 @@ func WithHealthEndpointAt(healthCheckClient grpc_health_v1.HealthClient, endpoin
w.Header().Set("Content-Type", "application/json")
if resp.GetStatus() != grpc_health_v1.HealthCheckResponse_SERVING {
var err error
switch resp.GetStatus() {
case grpc_health_v1.HealthCheckResponse_NOT_SERVING, grpc_health_v1.HealthCheckResponse_UNKNOWN:
err = status.Error(codes.Unavailable, resp.String())
@@ -322,17 +320,6 @@ func (s *ServeMux) ServeHTTP(w http.ResponseWriter, r *http.Request) {
path = r.URL.RawPath
}
var components []string
// since in UnescapeModeLegacy, the URL will already have been fully unescaped, if we also split on "%2F"
// in this escaping mode we would be double unescaping but in UnescapingModeAllCharacters, we still do as the
// path is the RawPath (i.e. unescaped). That does mean that the behavior of this function will change its default
// behavior when the UnescapingModeDefault gets changed from UnescapingModeLegacy to UnescapingModeAllExceptReserved
if s.unescapingMode == UnescapingModeAllCharacters {
components = encodedPathSplitter.Split(path[1:], -1)
} else {
components = strings.Split(path[1:], "/")
}
if override := r.Header.Get("X-HTTP-Method-Override"); override != "" && s.isPathLengthFallback(r) {
r.Method = strings.ToUpper(override)
if err := r.ParseForm(); err != nil {
@@ -343,8 +330,18 @@ func (s *ServeMux) ServeHTTP(w http.ResponseWriter, r *http.Request) {
}
}
// Verb out here is to memoize for the fallback case below
var verb string
var pathComponents []string
// since in UnescapeModeLegacy, the URL will already have been fully unescaped, if we also split on "%2F"
// in this escaping mode we would be double unescaping but in UnescapingModeAllCharacters, we still do as the
// path is the RawPath (i.e. unescaped). That does mean that the behavior of this function will change its default
// behavior when the UnescapingModeDefault gets changed from UnescapingModeLegacy to UnescapingModeAllExceptReserved
if s.unescapingMode == UnescapingModeAllCharacters {
pathComponents = encodedPathSplitter.Split(path[1:], -1)
} else {
pathComponents = strings.Split(path[1:], "/")
}
lastPathComponent := pathComponents[len(pathComponents)-1]
for _, h := range s.handlers[r.Method] {
// If the pattern has a verb, explicitly look for a suffix in the last
@@ -355,23 +352,28 @@ func (s *ServeMux) ServeHTTP(w http.ResponseWriter, r *http.Request) {
// parser because we know what verb we're looking for, however, there
// are still some cases that the parser itself cannot disambiguate. See
// the comment there if interested.
var verb string
patVerb := h.pat.Verb()
l := len(components)
lastComponent := components[l-1]
var idx int = -1
if patVerb != "" && strings.HasSuffix(lastComponent, ":"+patVerb) {
idx = len(lastComponent) - len(patVerb) - 1
idx := -1
if patVerb != "" && strings.HasSuffix(lastPathComponent, ":"+patVerb) {
idx = len(lastPathComponent) - len(patVerb) - 1
}
if idx == 0 {
_, outboundMarshaler := MarshalerForRequest(s, r)
s.routingErrorHandler(ctx, s, outboundMarshaler, w, r, http.StatusNotFound)
return
}
comps := make([]string, len(pathComponents))
copy(comps, pathComponents)
if idx > 0 {
components[l-1], verb = lastComponent[:idx], lastComponent[idx+1:]
comps[len(comps)-1], verb = lastPathComponent[:idx], lastPathComponent[idx+1:]
}
pathParams, err := h.pat.MatchAndEscape(components, verb, s.unescapingMode)
pathParams, err := h.pat.MatchAndEscape(comps, verb, s.unescapingMode)
if err != nil {
var mse MalformedSequenceError
if ok := errors.As(err, &mse); ok {
@@ -387,14 +389,33 @@ func (s *ServeMux) ServeHTTP(w http.ResponseWriter, r *http.Request) {
return
}
// lookup other methods to handle fallback from GET to POST and
// to determine if it is NotImplemented or NotFound.
// if no handler has found for the request, lookup for other methods
// to handle POST -> GET fallback if the request is subject to path
// length fallback.
// Note we are not eagerly checking the request here as we want to return the
// right HTTP status code, and we need to process the fallback candidates in
// order to do that.
for m, handlers := range s.handlers {
if m == r.Method {
continue
}
for _, h := range handlers {
pathParams, err := h.pat.MatchAndEscape(components, verb, s.unescapingMode)
var verb string
patVerb := h.pat.Verb()
idx := -1
if patVerb != "" && strings.HasSuffix(lastPathComponent, ":"+patVerb) {
idx = len(lastPathComponent) - len(patVerb) - 1
}
comps := make([]string, len(pathComponents))
copy(comps, pathComponents)
if idx > 0 {
comps[len(comps)-1], verb = lastPathComponent[:idx], lastPathComponent[idx+1:]
}
pathParams, err := h.pat.MatchAndEscape(comps, verb, s.unescapingMode)
if err != nil {
var mse MalformedSequenceError
if ok := errors.As(err, &mse); ok {
@@ -406,8 +427,11 @@ func (s *ServeMux) ServeHTTP(w http.ResponseWriter, r *http.Request) {
}
continue
}
// X-HTTP-Method-Override is optional. Always allow fallback to POST.
if s.isPathLengthFallback(r) {
// Also, only consider POST -> GET fallbacks, and avoid falling back to
// potentially dangerous operations like DELETE.
if s.isPathLengthFallback(r) && m == http.MethodGet {
if err := r.ParseForm(); err != nil {
_, outboundMarshaler := MarshalerForRequest(s, r)
sterr := status.Error(codes.InvalidArgument, err.Error())

2
vendor/github.com/grpc-ecosystem/grpc-gateway/v2/runtime/pattern.go generated vendored
View File

@@ -15,8 +15,6 @@ var (
ErrNotMatch = errors.New("not match to the path pattern")
// ErrInvalidPattern indicates that the given definition of Pattern is not valid.
ErrInvalidPattern = errors.New("invalid pattern")
// ErrMalformedSequence indicates that an escape sequence was malformed.
ErrMalformedSequence = errors.New("malformed escape sequence")
)
type MalformedSequenceError string

40
vendor/github.com/grpc-ecosystem/grpc-gateway/v2/runtime/query.go generated vendored
View File

@@ -10,13 +10,13 @@ import (
"time"
"github.com/grpc-ecosystem/grpc-gateway/v2/utilities"
"google.golang.org/genproto/protobuf/field_mask"
"google.golang.org/grpc/grpclog"
"google.golang.org/protobuf/encoding/protojson"
"google.golang.org/protobuf/proto"
"google.golang.org/protobuf/reflect/protoreflect"
"google.golang.org/protobuf/reflect/protoregistry"
"google.golang.org/protobuf/types/known/durationpb"
field_mask "google.golang.org/protobuf/types/known/fieldmaskpb"
"google.golang.org/protobuf/types/known/structpb"
"google.golang.org/protobuf/types/known/timestamppb"
"google.golang.org/protobuf/types/known/wrapperspb"
@@ -47,8 +47,7 @@ type DefaultQueryParser struct{}
// A value is ignored if its key starts with one of the elements in "filter".
func (*DefaultQueryParser) Parse(msg proto.Message, values url.Values, filter *utilities.DoubleArray) error {
for key, values := range values {
match := valuesKeyRegexp.FindStringSubmatch(key)
if len(match) == 3 {
if match := valuesKeyRegexp.FindStringSubmatch(key); len(match) == 3 {
key = match[1]
values = append([]string{match[2]}, values...)
}
@@ -180,10 +179,10 @@ func parseField(fieldDescriptor protoreflect.FieldDescriptor, value string) (pro
return protoreflect.ValueOfBool(v), nil
case protoreflect.EnumKind:
enum, err := protoregistry.GlobalTypes.FindEnumByName(fieldDescriptor.Enum().FullName())
switch {
case errors.Is(err, protoregistry.NotFound):
return protoreflect.Value{}, fmt.Errorf("enum %q is not registered", fieldDescriptor.Enum().FullName())
case err != nil:
if err != nil {
if errors.Is(err, protoregistry.NotFound) {
return protoreflect.Value{}, fmt.Errorf("enum %q is not registered", fieldDescriptor.Enum().FullName())
}
return protoreflect.Value{}, fmt.Errorf("failed to look up enum: %w", err)
}
// Look for enum by name
@@ -194,8 +193,7 @@ func parseField(fieldDescriptor protoreflect.FieldDescriptor, value string) (pro
return protoreflect.Value{}, fmt.Errorf("%q is not a valid value", value)
}
// Look for enum by number
v = enum.Descriptor().Values().ByNumber(protoreflect.EnumNumber(i))
if v == nil {
if v = enum.Descriptor().Values().ByNumber(protoreflect.EnumNumber(i)); v == nil {
return protoreflect.Value{}, fmt.Errorf("%q is not a valid value", value)
}
}
@@ -271,66 +269,64 @@ func parseMessage(msgDescriptor protoreflect.MessageDescriptor, value string) (p
if err != nil {
return protoreflect.Value{}, err
}
msg = &wrapperspb.DoubleValue{Value: v}
msg = wrapperspb.Double(v)
case "google.protobuf.FloatValue":
v, err := strconv.ParseFloat(value, 32)
if err != nil {
return protoreflect.Value{}, err
}
msg = &wrapperspb.FloatValue{Value: float32(v)}
msg = wrapperspb.Float(float32(v))
case "google.protobuf.Int64Value":
v, err := strconv.ParseInt(value, 10, 64)
if err != nil {
return protoreflect.Value{}, err
}
msg = &wrapperspb.Int64Value{Value: v}
msg = wrapperspb.Int64(v)
case "google.protobuf.Int32Value":
v, err := strconv.ParseInt(value, 10, 32)
if err != nil {
return protoreflect.Value{}, err
}
msg = &wrapperspb.Int32Value{Value: int32(v)}
msg = wrapperspb.Int32(int32(v))
case "google.protobuf.UInt64Value":
v, err := strconv.ParseUint(value, 10, 64)
if err != nil {
return protoreflect.Value{}, err
}
msg = &wrapperspb.UInt64Value{Value: v}
msg = wrapperspb.UInt64(v)
case "google.protobuf.UInt32Value":
v, err := strconv.ParseUint(value, 10, 32)
if err != nil {
return protoreflect.Value{}, err
}
msg = &wrapperspb.UInt32Value{Value: uint32(v)}
msg = wrapperspb.UInt32(uint32(v))
case "google.protobuf.BoolValue":
v, err := strconv.ParseBool(value)
if err != nil {
return protoreflect.Value{}, err
}
msg = &wrapperspb.BoolValue{Value: v}
msg = wrapperspb.Bool(v)
case "google.protobuf.StringValue":
msg = &wrapperspb.StringValue{Value: value}
msg = wrapperspb.String(value)
case "google.protobuf.BytesValue":
v, err := Bytes(value)
if err != nil {
return protoreflect.Value{}, err
}
msg = &wrapperspb.BytesValue{Value: v}
msg = wrapperspb.Bytes(v)
case "google.protobuf.FieldMask":
fm := &field_mask.FieldMask{}
fm.Paths = append(fm.Paths, strings.Split(value, ",")...)
msg = fm
case "google.protobuf.Value":
var v structpb.Value
err := protojson.Unmarshal([]byte(value), &v)
if err != nil {
if err := protojson.Unmarshal([]byte(value), &v); err != nil {
return protoreflect.Value{}, err
}
msg = &v
case "google.protobuf.Struct":
var v structpb.Struct
err := protojson.Unmarshal([]byte(value), &v)
if err != nil {
if err := protojson.Unmarshal([]byte(value), &v); err != nil {
return protoreflect.Value{}, err
}
msg = &v

3
vendor/github.com/grpc-ecosystem/grpc-gateway/v2/utilities/readerfactory.go generated vendored
View File

@@ -3,13 +3,12 @@ package utilities
import (
"bytes"
"io"
"io/ioutil"
)
// IOReaderFactory takes in an io.Reader and returns a function that will allow you to create a new reader that begins
// at the start of the stream
func IOReaderFactory(r io.Reader) (func() io.Reader, error) {
b, err := ioutil.ReadAll(r)
b, err := io.ReadAll(r)
if err != nil {
return nil, err
}

2
vendor/github.com/grpc-ecosystem/grpc-gateway/v2/utilities/trie.go generated vendored
View File

@@ -40,7 +40,7 @@ func NewDoubleArray(seqs [][]string) *DoubleArray {
func registerTokens(da *DoubleArray, seqs [][]string) [][]int {
var result [][]int
for _, seq := range seqs {
var encoded []int
encoded := make([]int, 0, len(seq))
for _, token := range seq {
if _, ok := da.Encoding[token]; !ok {
da.Encoding[token] = len(da.Encoding)

124
vendor/go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc/grpctrace.go → vendor/go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc/config.go generated vendored
View File

@@ -15,14 +15,11 @@
package otelgrpc // import "go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"
import (
"context"
"google.golang.org/grpc/metadata"
"go.opentelemetry.io/otel"
"go.opentelemetry.io/otel/attribute"
"go.opentelemetry.io/otel/baggage"
"go.opentelemetry.io/otel/metric"
"go.opentelemetry.io/otel/propagation"
semconv "go.opentelemetry.io/otel/semconv/v1.17.0"
"go.opentelemetry.io/otel/trace"
)
@@ -40,9 +37,17 @@ type Filter func(*InterceptorInfo) bool
// config is a group of options for this instrumentation.
type config struct {
Filter Filter
Propagators propagation.TextMapPropagator
TracerProvider trace.TracerProvider
Filter Filter
Propagators propagation.TextMapPropagator
TracerProvider trace.TracerProvider
MeterProvider metric.MeterProvider
SpanStartOptions []trace.SpanStartOption
ReceivedEvent bool
SentEvent bool
meter metric.Meter
rpcServerDuration metric.Int64Histogram
}
// Option applies an option value for a config.
@@ -55,10 +60,25 @@ func newConfig(opts []Option) *config {
c := &config{
Propagators: otel.GetTextMapPropagator(),
TracerProvider: otel.GetTracerProvider(),
MeterProvider: otel.GetMeterProvider(),
}
for _, o := range opts {
o.apply(c)
}
c.meter = c.MeterProvider.Meter(
instrumentationName,
metric.WithInstrumentationVersion(Version()),
metric.WithSchemaURL(semconv.SchemaURL),
)
var err error
c.rpcServerDuration, err = c.meter.Int64Histogram("rpc.server.duration",
metric.WithDescription("Measures the duration of inbound RPC."),
metric.WithUnit("ms"))
if err != nil {
otel.Handle(err)
}
return c
}
@@ -105,59 +125,63 @@ func WithTracerProvider(tp trace.TracerProvider) Option {
return tracerProviderOption{tp: tp}
}
type metadataSupplier struct {
metadata *metadata.MD
}
type meterProviderOption struct{ mp metric.MeterProvider }
// assert that metadataSupplier implements the TextMapCarrier interface.
var _ propagation.TextMapCarrier = &metadataSupplier{}
func (s *metadataSupplier) Get(key string) string {
values := s.metadata.Get(key)
if len(values) == 0 {
return ""
func (o meterProviderOption) apply(c *config) {
if o.mp != nil {
c.MeterProvider = o.mp
}
return values[0]
}
func (s *metadataSupplier) Set(key string, value string) {
s.metadata.Set(key, value)
// WithMeterProvider returns an Option to use the MeterProvider when
// creating a Meter. If this option is not provide the global MeterProvider will be used.
func WithMeterProvider(mp metric.MeterProvider) Option {
return meterProviderOption{mp: mp}
}
func (s *metadataSupplier) Keys() []string {
out := make([]string, 0, len(*s.metadata))
for key := range *s.metadata {
out = append(out, key)
// Event type that can be recorded, see WithMessageEvents.
type Event int
// Different types of events that can be recorded, see WithMessageEvents.
const (
ReceivedEvents Event = iota
SentEvents
)
type messageEventsProviderOption struct {
events []Event
}
func (m messageEventsProviderOption) apply(c *config) {
for _, e := range m.events {
switch e {
case ReceivedEvents:
c.ReceivedEvent = true
case SentEvents:
c.SentEvent = true
}
}
return out
}
// Inject injects correlation context and span context into the gRPC
// metadata object. This function is meant to be used on outgoing
// requests.
func Inject(ctx context.Context, md *metadata.MD, opts ...Option) {
c := newConfig(opts)
inject(ctx, md, c.Propagators)
// WithMessageEvents configures the Handler to record the specified events
// (span.AddEvent) on spans. By default only summary attributes are added at the
// end of the request.
//
// Valid events are:
// - ReceivedEvents: Record the number of bytes read after every gRPC read operation.
// - SentEvents: Record the number of bytes written after every gRPC write operation.
func WithMessageEvents(events ...Event) Option {
return messageEventsProviderOption{events: events}
}
func inject(ctx context.Context, md *metadata.MD, propagators propagation.TextMapPropagator) {
propagators.Inject(ctx, &metadataSupplier{
metadata: md,
})
type spanStartOption struct{ opts []trace.SpanStartOption }
func (o spanStartOption) apply(c *config) {
c.SpanStartOptions = append(c.SpanStartOptions, o.opts...)
}
// Extract returns the correlation context and span context that
// another service encoded in the gRPC metadata object with Inject.
// This function is meant to be used on incoming requests.
func Extract(ctx context.Context, md *metadata.MD, opts ...Option) (baggage.Baggage, trace.SpanContext) {
c := newConfig(opts)
return extract(ctx, md, c.Propagators)
}
func extract(ctx context.Context, md *metadata.MD, propagators propagation.TextMapPropagator) (baggage.Baggage, trace.SpanContext) {
ctx = propagators.Extract(ctx, &metadataSupplier{
metadata: md,
})
return baggage.FromContext(ctx), trace.SpanContextFromContext(ctx)
// WithSpanOptions configures an additional set of
// trace.SpanOptions, which are applied to each new span.
func WithSpanOptions(opts ...trace.SpanStartOption) Option {
return spanStartOption{opts}
}

45
vendor/go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc/doc.go generated vendored Normal file
View File

@@ -0,0 +1,45 @@
// Copyright The OpenTelemetry Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
/*
Package otelgrpc is the instrumentation library for [google.golang.org/grpc]
For now you can instrument your program which use [google.golang.org/grpc] in two ways:
- by [grpc.UnaryClientInterceptor], [grpc.UnaryServerInterceptor], [grpc.StreamClientInterceptor], [grpc.StreamServerInterceptor]
- by [stats.Handler]
Notice: Do not use both interceptors and [stats.Handler] at the same time! If so, you will get duplicated spans and the parent/child relationships between spans will also be broken.
We strongly still recommand you to use [stats.Handler], mainly for two reasons:
Functional advantages: [stats.Handler] has more information for user to build more flexible and granular metric, for example
- multiple different types of represent "data length": In [stats.InPayload], there exists "Length", "CompressedLength", "WireLength" to denote the size of uncompressed, compressed payload data, with or without framing data. But in interceptors, we can only got uncompressed data, and this feature is also removed due to performance problem.
- more accurate timestamp: [stats.InPayload]'s "RecvTime" and [stats.OutPayload]'s "SentTime" records more accurate timestamp that server got and sent the message, the timestamp recorded by interceptors depends on the location of this interceptors in the total interceptor chain.
- some other use cases: for example, catch failure of decoding message.
Performance advantages: If too many interceptors are registered in a service, the interceptor chain can become too long, which increases the latency and processing time of the entire RPC call.
[stats.Handler]: https://pkg.go.dev/google.golang.org/grpc/stats#Handler
[grpc.UnaryClientInterceptor]: https://pkg.go.dev/google.golang.org/grpc#UnaryClientInterceptor
[grpc.UnaryServerInterceptor]: https://pkg.go.dev/google.golang.org/grpc#UnaryServerInterceptor
[grpc.StreamClientInterceptor]: https://pkg.go.dev/google.golang.org/grpc#StreamClientInterceptor
[grpc.StreamServerInterceptor]: https://pkg.go.dev/google.golang.org/grpc#StreamServerInterceptor
[stats.OutPayload]: https://pkg.go.dev/google.golang.org/grpc/stats#OutPayload
[stats.InPayload]: https://pkg.go.dev/google.golang.org/grpc/stats#InPayload
*/
package otelgrpc // import "go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"

274
vendor/go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc/interceptor.go generated vendored
View File

@@ -20,41 +20,37 @@ import (
"context"
"io"
"net"
"github.com/golang/protobuf/proto" // nolint:staticcheck
"strconv"
"time"
"google.golang.org/grpc"
grpc_codes "google.golang.org/grpc/codes"
"google.golang.org/grpc/metadata"
"google.golang.org/grpc/peer"
"google.golang.org/grpc/status"
"google.golang.org/protobuf/proto"
"go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc/internal"
"go.opentelemetry.io/otel/attribute"
"go.opentelemetry.io/otel/baggage"
"go.opentelemetry.io/otel/codes"
semconv "go.opentelemetry.io/otel/semconv/v1.12.0"
"go.opentelemetry.io/otel/metric"
semconv "go.opentelemetry.io/otel/semconv/v1.17.0"
"go.opentelemetry.io/otel/trace"
)
type messageType attribute.KeyValue
// Event adds an event of the messageType to the span associated with the
// passed context with id and size (if message is a proto message).
func (m messageType) Event(ctx context.Context, id int, message interface{}) {
// passed context with a message id.
func (m messageType) Event(ctx context.Context, id int, _ interface{}) {
span := trace.SpanFromContext(ctx)
if p, ok := message.(proto.Message); ok {
span.AddEvent("message", trace.WithAttributes(
attribute.KeyValue(m),
RPCMessageIDKey.Int(id),
RPCMessageUncompressedSizeKey.Int(proto.Size(p)),
))
} else {
span.AddEvent("message", trace.WithAttributes(
attribute.KeyValue(m),
RPCMessageIDKey.Int(id),
))
if !span.IsRecording() {
return
}
span.AddEvent("message", trace.WithAttributes(
attribute.KeyValue(m),
RPCMessageIDKey.Int(id),
))
}
var (
@@ -66,6 +62,11 @@ var (
// for use in a grpc.Dial call.
func UnaryClientInterceptor(opts ...Option) grpc.UnaryClientInterceptor {
cfg := newConfig(opts)
tracer := cfg.TracerProvider.Tracer(
instrumentationName,
trace.WithInstrumentationVersion(Version()),
)
return func(
ctx context.Context,
method string,
@@ -82,32 +83,32 @@ func UnaryClientInterceptor(opts ...Option) grpc.UnaryClientInterceptor {
return invoker(ctx, method, req, reply, cc, callOpts...)
}
requestMetadata, _ := metadata.FromOutgoingContext(ctx)
metadataCopy := requestMetadata.Copy()
name, attr := spanInfo(method, cc.Target())
tracer := cfg.TracerProvider.Tracer(
instrumentationName,
trace.WithInstrumentationVersion(SemVersion()),
startOpts := append([]trace.SpanStartOption{
trace.WithSpanKind(trace.SpanKindClient),
trace.WithAttributes(attr...)},
cfg.SpanStartOptions...,
)
name, attr := spanInfo(method, cc.Target())
var span trace.Span
ctx, span = tracer.Start(
ctx, span := tracer.Start(
ctx,
name,
trace.WithSpanKind(trace.SpanKindClient),
trace.WithAttributes(attr...),
startOpts...,
)
defer span.End()
inject(ctx, &metadataCopy, cfg.Propagators)
ctx = metadata.NewOutgoingContext(ctx, metadataCopy)
ctx = inject(ctx, cfg.Propagators)
messageSent.Event(ctx, 1, req)
if cfg.SentEvent {
messageSent.Event(ctx, 1, req)
}
err := invoker(ctx, method, req, reply, cc, callOpts...)
messageReceived.Event(ctx, 1, reply)
if cfg.ReceivedEvent {
messageReceived.Event(ctx, 1, reply)
}
if err != nil {
s, _ := status.FromError(err)
@@ -143,6 +144,9 @@ type clientStream struct {
eventsDone chan struct{}
finished chan error
receivedEvent bool
sentEvent bool
receivedMessageID int
sentMessageID int
}
@@ -160,7 +164,10 @@ func (w *clientStream) RecvMsg(m interface{}) error {
w.sendStreamEvent(errorEvent, err)
} else {
w.receivedMessageID++
messageReceived.Event(w.Context(), w.receivedMessageID, m)
if w.receivedEvent {
messageReceived.Event(w.Context(), w.receivedMessageID, m)
}
}
return err
@@ -170,7 +177,10 @@ func (w *clientStream) SendMsg(m interface{}) error {
err := w.ClientStream.SendMsg(m)
w.sentMessageID++
messageSent.Event(w.Context(), w.sentMessageID, m)
if w.sentEvent {
messageSent.Event(w.Context(), w.sentMessageID, m)
}
if err != nil {
w.sendStreamEvent(errorEvent, err)
@@ -199,7 +209,7 @@ func (w *clientStream) CloseSend() error {
return err
}
func wrapClientStream(ctx context.Context, s grpc.ClientStream, desc *grpc.StreamDesc) *clientStream {
func wrapClientStream(ctx context.Context, s grpc.ClientStream, desc *grpc.StreamDesc, cfg *config) *clientStream {
events := make(chan streamEvent)
eventsDone := make(chan struct{})
finished := make(chan error)
@@ -226,11 +236,13 @@ func wrapClientStream(ctx context.Context, s grpc.ClientStream, desc *grpc.Strea
}()
return &clientStream{
ClientStream: s,
desc: desc,
events: events,
eventsDone: eventsDone,
finished: finished,
ClientStream: s,
desc: desc,
events: events,
eventsDone: eventsDone,
finished: finished,
receivedEvent: cfg.ReceivedEvent,
sentEvent: cfg.SentEvent,
}
}
@@ -245,6 +257,11 @@ func (w *clientStream) sendStreamEvent(eventType streamEventType, err error) {
// for use in a grpc.Dial call.
func StreamClientInterceptor(opts ...Option) grpc.StreamClientInterceptor {
cfg := newConfig(opts)
tracer := cfg.TracerProvider.Tracer(
instrumentationName,
trace.WithInstrumentationVersion(Version()),
)
return func(
ctx context.Context,
desc *grpc.StreamDesc,
@@ -261,25 +278,21 @@ func StreamClientInterceptor(opts ...Option) grpc.StreamClientInterceptor {
return streamer(ctx, desc, cc, method, callOpts...)
}
requestMetadata, _ := metadata.FromOutgoingContext(ctx)
metadataCopy := requestMetadata.Copy()
name, attr := spanInfo(method, cc.Target())
tracer := cfg.TracerProvider.Tracer(
instrumentationName,
trace.WithInstrumentationVersion(SemVersion()),
startOpts := append([]trace.SpanStartOption{
trace.WithSpanKind(trace.SpanKindClient),
trace.WithAttributes(attr...)},
cfg.SpanStartOptions...,
)
name, attr := spanInfo(method, cc.Target())
var span trace.Span
ctx, span = tracer.Start(
ctx, span := tracer.Start(
ctx,
name,
trace.WithSpanKind(trace.SpanKindClient),
trace.WithAttributes(attr...),
startOpts...,
)
inject(ctx, &metadataCopy, cfg.Propagators)
ctx = metadata.NewOutgoingContext(ctx, metadataCopy)
ctx = inject(ctx, cfg.Propagators)
s, err := streamer(ctx, desc, cc, method, callOpts...)
if err != nil {
@@ -289,7 +302,7 @@ func StreamClientInterceptor(opts ...Option) grpc.StreamClientInterceptor {
span.End()
return s, err
}
stream := wrapClientStream(ctx, s, desc)
stream := wrapClientStream(ctx, s, desc, cfg)
go func() {
err := <-stream.finished
@@ -313,6 +326,11 @@ func StreamClientInterceptor(opts ...Option) grpc.StreamClientInterceptor {
// for use in a grpc.NewServer call.
func UnaryServerInterceptor(opts ...Option) grpc.UnaryServerInterceptor {
cfg := newConfig(opts)
tracer := cfg.TracerProvider.Tracer(
instrumentationName,
trace.WithInstrumentationVersion(Version()),
)
return func(
ctx context.Context,
req interface{},
@@ -327,37 +345,49 @@ func UnaryServerInterceptor(opts ...Option) grpc.UnaryServerInterceptor {
return handler(ctx, req)
}
requestMetadata, _ := metadata.FromIncomingContext(ctx)
metadataCopy := requestMetadata.Copy()
ctx = extract(ctx, cfg.Propagators)
name, attr := spanInfo(info.FullMethod, peerFromCtx(ctx))
bags, spanCtx := Extract(ctx, &metadataCopy, opts...)
ctx = baggage.ContextWithBaggage(ctx, bags)
tracer := cfg.TracerProvider.Tracer(
instrumentationName,
trace.WithInstrumentationVersion(SemVersion()),
startOpts := append([]trace.SpanStartOption{
trace.WithSpanKind(trace.SpanKindServer),
trace.WithAttributes(attr...)},
cfg.SpanStartOptions...,
)
name, attr := spanInfo(info.FullMethod, peerFromCtx(ctx))
ctx, span := tracer.Start(
trace.ContextWithRemoteSpanContext(ctx, spanCtx),
trace.ContextWithRemoteSpanContext(ctx, trace.SpanContextFromContext(ctx)),
name,
trace.WithSpanKind(trace.SpanKindServer),
trace.WithAttributes(attr...),
startOpts...,
)
defer span.End()
messageReceived.Event(ctx, 1, req)
if cfg.ReceivedEvent {
messageReceived.Event(ctx, 1, req)
}
var statusCode grpc_codes.Code
defer func(t time.Time) {
elapsedTime := time.Since(t) / time.Millisecond
attr = append(attr, semconv.RPCGRPCStatusCodeKey.Int64(int64(statusCode)))
o := metric.WithAttributes(attr...)
cfg.rpcServerDuration.Record(ctx, int64(elapsedTime), o)
}(time.Now())
resp, err := handler(ctx, req)
if err != nil {
s, _ := status.FromError(err)
span.SetStatus(codes.Error, s.Message())
statusCode, msg := serverStatus(s)
span.SetStatus(statusCode, msg)
span.SetAttributes(statusCodeAttr(s.Code()))
messageSent.Event(ctx, 1, s.Proto())
if cfg.SentEvent {
messageSent.Event(ctx, 1, s.Proto())
}
} else {
statusCode = grpc_codes.OK
span.SetAttributes(statusCodeAttr(grpc_codes.OK))
messageSent.Event(ctx, 1, resp)
if cfg.SentEvent {
messageSent.Event(ctx, 1, resp)
}
}
return resp, err
@@ -372,6 +402,9 @@ type serverStream struct {
receivedMessageID int
sentMessageID int
receivedEvent bool
sentEvent bool
}
func (w *serverStream) Context() context.Context {
@@ -383,7 +416,9 @@ func (w *serverStream) RecvMsg(m interface{}) error {
if err == nil {
w.receivedMessageID++
messageReceived.Event(w.Context(), w.receivedMessageID, m)
if w.receivedEvent {
messageReceived.Event(w.Context(), w.receivedMessageID, m)
}
}
return err
@@ -393,15 +428,19 @@ func (w *serverStream) SendMsg(m interface{}) error {
err := w.ServerStream.SendMsg(m)
w.sentMessageID++
messageSent.Event(w.Context(), w.sentMessageID, m)
if w.sentEvent {
messageSent.Event(w.Context(), w.sentMessageID, m)
}
return err
}
func wrapServerStream(ctx context.Context, ss grpc.ServerStream) *serverStream {
func wrapServerStream(ctx context.Context, ss grpc.ServerStream, cfg *config) *serverStream {
return &serverStream{
ServerStream: ss,
ctx: ctx,
ServerStream: ss,
ctx: ctx,
receivedEvent: cfg.ReceivedEvent,
sentEvent: cfg.SentEvent,
}
}
@@ -409,6 +448,11 @@ func wrapServerStream(ctx context.Context, ss grpc.ServerStream) *serverStream {
// for use in a grpc.NewServer call.
func StreamServerInterceptor(opts ...Option) grpc.StreamServerInterceptor {
cfg := newConfig(opts)
tracer := cfg.TracerProvider.Tracer(
instrumentationName,
trace.WithInstrumentationVersion(Version()),
)
return func(
srv interface{},
ss grpc.ServerStream,
@@ -421,34 +465,30 @@ func StreamServerInterceptor(opts ...Option) grpc.StreamServerInterceptor {
Type: StreamServer,
}
if cfg.Filter != nil && !cfg.Filter(i) {
return handler(srv, wrapServerStream(ctx, ss))
return handler(srv, wrapServerStream(ctx, ss, cfg))
}
requestMetadata, _ := metadata.FromIncomingContext(ctx)
metadataCopy := requestMetadata.Copy()
ctx = extract(ctx, cfg.Propagators)
name, attr := spanInfo(info.FullMethod, peerFromCtx(ctx))
bags, spanCtx := Extract(ctx, &metadataCopy, opts...)
ctx = baggage.ContextWithBaggage(ctx, bags)
tracer := cfg.TracerProvider.Tracer(
instrumentationName,
trace.WithInstrumentationVersion(SemVersion()),
startOpts := append([]trace.SpanStartOption{
trace.WithSpanKind(trace.SpanKindServer),
trace.WithAttributes(attr...)},
cfg.SpanStartOptions...,
)
name, attr := spanInfo(info.FullMethod, peerFromCtx(ctx))
ctx, span := tracer.Start(
trace.ContextWithRemoteSpanContext(ctx, spanCtx),
trace.ContextWithRemoteSpanContext(ctx, trace.SpanContextFromContext(ctx)),
name,
trace.WithSpanKind(trace.SpanKindServer),
trace.WithAttributes(attr...),
startOpts...,
)
defer span.End()
err := handler(srv, wrapServerStream(ctx, ss))
err := handler(srv, wrapServerStream(ctx, ss, cfg))
if err != nil {
s, _ := status.FromError(err)
span.SetStatus(codes.Error, s.Message())
statusCode, msg := serverStatus(s)
span.SetStatus(statusCode, msg)
span.SetAttributes(statusCodeAttr(s.Code()))
} else {
span.SetAttributes(statusCodeAttr(grpc_codes.OK))
@@ -461,28 +501,45 @@ func StreamServerInterceptor(opts ...Option) grpc.StreamServerInterceptor {
// spanInfo returns a span name and all appropriate attributes from the gRPC
// method and peer address.
func spanInfo(fullMethod, peerAddress string) (string, []attribute.KeyValue) {
attrs := []attribute.KeyValue{RPCSystemGRPC}
name, mAttrs := internal.ParseFullMethod(fullMethod)
peerAttrs := peerAttr(peerAddress)
attrs := make([]attribute.KeyValue, 0, 1+len(mAttrs)+len(peerAttrs))
attrs = append(attrs, RPCSystemGRPC)
attrs = append(attrs, mAttrs...)
attrs = append(attrs, peerAttr(peerAddress)...)
attrs = append(attrs, peerAttrs...)
return name, attrs
}
// peerAttr returns attributes about the peer address.
func peerAttr(addr string) []attribute.KeyValue {
host, port, err := net.SplitHostPort(addr)
host, p, err := net.SplitHostPort(addr)
if err != nil {
return []attribute.KeyValue(nil)
return nil
}
if host == "" {
host = "127.0.0.1"
}
return []attribute.KeyValue{
semconv.NetPeerIPKey.String(host),
semconv.NetPeerPortKey.String(port),
port, err := strconv.Atoi(p)
if err != nil {
return nil
}
var attr []attribute.KeyValue
if ip := net.ParseIP(host); ip != nil {
attr = []attribute.KeyValue{
semconv.NetSockPeerAddr(host),
semconv.NetSockPeerPort(port),
}
} else {
attr = []attribute.KeyValue{
semconv.NetPeerName(host),
semconv.NetPeerPort(port),
}
}
return attr
}
// peerFromCtx returns a peer address from a context, if one exists.
@@ -498,3 +555,26 @@ func peerFromCtx(ctx context.Context) string {
func statusCodeAttr(c grpc_codes.Code) attribute.KeyValue {
return GRPCStatusCodeKey.Int64(int64(c))
}
// serverStatus returns a span status code and message for a given gRPC
// status code. It maps specific gRPC status codes to a corresponding span
// status code and message. This function is intended for use on the server
// side of a gRPC connection.
//
// If the gRPC status code is Unknown, DeadlineExceeded, Unimplemented,
// Internal, Unavailable, or DataLoss, it returns a span status code of Error
// and the message from the gRPC status. Otherwise, it returns a span status
// code of Unset and an empty message.
func serverStatus(grpcStatus *status.Status) (codes.Code, string) {
switch grpcStatus.Code() {
case grpc_codes.Unknown,
grpc_codes.DeadlineExceeded,
grpc_codes.Unimplemented,
grpc_codes.Internal,
grpc_codes.Unavailable,
grpc_codes.DataLoss:
return codes.Error, grpcStatus.Message()
default:
return codes.Unset, ""
}
}

26
vendor/go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc/internal/parse.go generated vendored
View File

@@ -18,26 +18,34 @@ import (
"strings"
"go.opentelemetry.io/otel/attribute"
semconv "go.opentelemetry.io/otel/semconv/v1.12.0"
semconv "go.opentelemetry.io/otel/semconv/v1.17.0"
)
// ParseFullMethod returns a span name following the OpenTelemetry semantic
// conventions as well as all applicable span attribute.KeyValue attributes based
// on a gRPC's FullMethod.
//
// Parsing is consistent with grpc-go implementation:
// https://github.com/grpc/grpc-go/blob/v1.57.0/internal/grpcutil/method.go#L26-L39
func ParseFullMethod(fullMethod string) (string, []attribute.KeyValue) {
name := strings.TrimLeft(fullMethod, "/")
parts := strings.SplitN(name, "/", 2)
if len(parts) != 2 {
if !strings.HasPrefix(fullMethod, "/") {
// Invalid format, does not follow `/package.service/method`.
return name, []attribute.KeyValue(nil)
return fullMethod, nil
}
name := fullMethod[1:]
pos := strings.LastIndex(name, "/")
if pos < 0 {
// Invalid format, does not follow `/package.service/method`.
return name, nil
}
service, method := name[:pos], name[pos+1:]
var attrs []attribute.KeyValue
if service := parts[0]; service != "" {
attrs = append(attrs, semconv.RPCServiceKey.String(service))
if service != "" {
attrs = append(attrs, semconv.RPCService(service))
}
if method := parts[1]; method != "" {
attrs = append(attrs, semconv.RPCMethodKey.String(method))
if method != "" {
attrs = append(attrs, semconv.RPCMethod(method))
}
return name, attrs
}

98
vendor/go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc/metadata_supplier.go generated vendored Normal file
View File

@@ -0,0 +1,98 @@
// Copyright The OpenTelemetry Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package otelgrpc // import "go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"
import (
"context"
"google.golang.org/grpc/metadata"
"go.opentelemetry.io/otel/baggage"
"go.opentelemetry.io/otel/propagation"
"go.opentelemetry.io/otel/trace"
)
type metadataSupplier struct {
metadata *metadata.MD
}
// assert that metadataSupplier implements the TextMapCarrier interface.
var _ propagation.TextMapCarrier = &metadataSupplier{}
func (s *metadataSupplier) Get(key string) string {
values := s.metadata.Get(key)
if len(values) == 0 {
return ""
}
return values[0]
}
func (s *metadataSupplier) Set(key string, value string) {
s.metadata.Set(key, value)
}
func (s *metadataSupplier) Keys() []string {
out := make([]string, 0, len(*s.metadata))
for key := range *s.metadata {
out = append(out, key)
}
return out
}
// Inject injects correlation context and span context into the gRPC
// metadata object. This function is meant to be used on outgoing
// requests.
// Deprecated: Unnecessary public func.
func Inject(ctx context.Context, md *metadata.MD, opts ...Option) {
c := newConfig(opts)
c.Propagators.Inject(ctx, &metadataSupplier{
metadata: md,
})
}
func inject(ctx context.Context, propagators propagation.TextMapPropagator) context.Context {
md, ok := metadata.FromOutgoingContext(ctx)
if !ok {
md = metadata.MD{}
}
propagators.Inject(ctx, &metadataSupplier{
metadata: &md,
})
return metadata.NewOutgoingContext(ctx, md)
}
// Extract returns the correlation context and span context that
// another service encoded in the gRPC metadata object with Inject.
// This function is meant to be used on incoming requests.
// Deprecated: Unnecessary public func.
func Extract(ctx context.Context, md *metadata.MD, opts ...Option) (baggage.Baggage, trace.SpanContext) {
c := newConfig(opts)
ctx = c.Propagators.Extract(ctx, &metadataSupplier{
metadata: md,
})
return baggage.FromContext(ctx), trace.SpanContextFromContext(ctx)
}
func extract(ctx context.Context, propagators propagation.TextMapPropagator) context.Context {
md, ok := metadata.FromIncomingContext(ctx)
if !ok {
md = metadata.MD{}
}
return propagators.Extract(ctx, &metadataSupplier{
metadata: &md,
})
}

4
vendor/go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc/semconv.go generated vendored
View File

@@ -16,7 +16,7 @@ package otelgrpc // import "go.opentelemetry.io/contrib/instrumentation/google.g
import (
"go.opentelemetry.io/otel/attribute"
semconv "go.opentelemetry.io/otel/semconv/v1.12.0"
semconv "go.opentelemetry.io/otel/semconv/v1.17.0"
)
// Semantic conventions for attribute keys for gRPC.
@@ -41,7 +41,7 @@ const (
// Semantic conventions for common RPC attributes.
var (
// Semantic convention for gRPC as the remoting system.
RPCSystemGRPC = semconv.RPCSystemKey.String("grpc")
RPCSystemGRPC = semconv.RPCSystemGRPC
// Semantic convention for a message named message.
RPCNameMessage = RPCNameKey.String("message")

187
vendor/go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc/stats_handler.go generated vendored Normal file
View File

@@ -0,0 +1,187 @@
// Copyright The OpenTelemetry Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package otelgrpc // import "go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"
import (
"context"
"sync/atomic"
grpc_codes "google.golang.org/grpc/codes"
"google.golang.org/grpc/stats"
"google.golang.org/grpc/status"
"go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc/internal"
"go.opentelemetry.io/otel/codes"
semconv "go.opentelemetry.io/otel/semconv/v1.17.0"
"go.opentelemetry.io/otel/trace"
)
type gRPCContextKey struct{}
type gRPCContext struct {
messagesReceived int64
messagesSent int64
}
// NewServerHandler creates a stats.Handler for gRPC server.
func NewServerHandler(opts ...Option) stats.Handler {
h := &serverHandler{
config: newConfig(opts),
}
h.tracer = h.config.TracerProvider.Tracer(
instrumentationName,
trace.WithInstrumentationVersion(SemVersion()),
)
return h
}
type serverHandler struct {
*config
tracer trace.Tracer
}
// TagRPC can attach some information to the given context.
func (h *serverHandler) TagRPC(ctx context.Context, info *stats.RPCTagInfo) context.Context {
ctx = extract(ctx, h.config.Propagators)
name, attrs := internal.ParseFullMethod(info.FullMethodName)
attrs = append(attrs, RPCSystemGRPC)
ctx, _ = h.tracer.Start(
trace.ContextWithRemoteSpanContext(ctx, trace.SpanContextFromContext(ctx)),
name,
trace.WithSpanKind(trace.SpanKindServer),
trace.WithAttributes(attrs...),
)
gctx := gRPCContext{}
return context.WithValue(ctx, gRPCContextKey{}, &gctx)
}
// HandleRPC processes the RPC stats.
func (h *serverHandler) HandleRPC(ctx context.Context, rs stats.RPCStats) {
handleRPC(ctx, rs)
}
// TagConn can attach some information to the given context.
func (h *serverHandler) TagConn(ctx context.Context, info *stats.ConnTagInfo) context.Context {
span := trace.SpanFromContext(ctx)
attrs := peerAttr(peerFromCtx(ctx))
span.SetAttributes(attrs...)
return ctx
}
// HandleConn processes the Conn stats.
func (h *serverHandler) HandleConn(ctx context.Context, info stats.ConnStats) {
}
// NewClientHandler creates a stats.Handler for gRPC client.
func NewClientHandler(opts ...Option) stats.Handler {
h := &clientHandler{
config: newConfig(opts),
}
h.tracer = h.config.TracerProvider.Tracer(
instrumentationName,
trace.WithInstrumentationVersion(SemVersion()),
)
return h
}
type clientHandler struct {
*config
tracer trace.Tracer
}
// TagRPC can attach some information to the given context.
func (h *clientHandler) TagRPC(ctx context.Context, info *stats.RPCTagInfo) context.Context {
name, attrs := internal.ParseFullMethod(info.FullMethodName)
attrs = append(attrs, RPCSystemGRPC)
ctx, _ = h.tracer.Start(
ctx,
name,
trace.WithSpanKind(trace.SpanKindClient),
trace.WithAttributes(attrs...),
)
gctx := gRPCContext{}
return inject(context.WithValue(ctx, gRPCContextKey{}, &gctx), h.config.Propagators)
}
// HandleRPC processes the RPC stats.
func (h *clientHandler) HandleRPC(ctx context.Context, rs stats.RPCStats) {
handleRPC(ctx, rs)
}
// TagConn can attach some information to the given context.
func (h *clientHandler) TagConn(ctx context.Context, cti *stats.ConnTagInfo) context.Context {
span := trace.SpanFromContext(ctx)
attrs := peerAttr(cti.RemoteAddr.String())
span.SetAttributes(attrs...)
return ctx
}
// HandleConn processes the Conn stats.
func (h *clientHandler) HandleConn(context.Context, stats.ConnStats) {
// no-op
}
func handleRPC(ctx context.Context, rs stats.RPCStats) {
span := trace.SpanFromContext(ctx)
gctx, _ := ctx.Value(gRPCContextKey{}).(*gRPCContext)
var messageId int64
switch rs := rs.(type) {
case *stats.Begin:
case *stats.InPayload:
if gctx != nil {
messageId = atomic.AddInt64(&gctx.messagesReceived, 1)
}
span.AddEvent("message",
trace.WithAttributes(
semconv.MessageTypeReceived,
semconv.MessageIDKey.Int64(messageId),
semconv.MessageCompressedSizeKey.Int(rs.CompressedLength),
semconv.MessageUncompressedSizeKey.Int(rs.Length),
),
)
case *stats.OutPayload:
if gctx != nil {
messageId = atomic.AddInt64(&gctx.messagesSent, 1)
}
span.AddEvent("message",
trace.WithAttributes(
semconv.MessageTypeSent,
semconv.MessageIDKey.Int64(messageId),
semconv.MessageCompressedSizeKey.Int(rs.CompressedLength),
semconv.MessageUncompressedSizeKey.Int(rs.Length),
),
)
case *stats.End:
if rs.Error != nil {
s, _ := status.FromError(rs.Error)
span.SetStatus(codes.Error, s.Message())
span.SetAttributes(statusCodeAttr(s.Code()))
} else {
span.SetAttributes(statusCodeAttr(grpc_codes.OK))
}
span.End()
default:
return
}
}

6
vendor/go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc/version.go generated vendored
View File

@@ -16,11 +16,13 @@ package otelgrpc // import "go.opentelemetry.io/contrib/instrumentation/google.g
// Version is the current release version of the gRPC instrumentation.
func Version() string {
return "0.35.0"
return "0.45.0"
// This string is updated by the pre_release.sh script during release
}
// SemVersion is the semantic version to be supplied to tracer/meter creation.
//
// Deprecated: Use [Version] instead.
func SemVersion() string {
return "semver:" + Version()
return Version()
}

2
vendor/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp/common.go generated vendored
View File

@@ -42,5 +42,5 @@ const (
type Filter func(*http.Request) bool
func newTracer(tp trace.TracerProvider) trace.Tracer {
return tp.Tracer(instrumentationName, trace.WithInstrumentationVersion(SemVersion()))
return tp.Tracer(instrumentationName, trace.WithInstrumentationVersion(Version()))
}

14
vendor/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp/config.go generated vendored
View File

@@ -21,7 +21,6 @@ import (
"go.opentelemetry.io/otel"
"go.opentelemetry.io/otel/metric"
"go.opentelemetry.io/otel/metric/global"
"go.opentelemetry.io/otel/propagation"
"go.opentelemetry.io/otel/trace"
)
@@ -33,6 +32,7 @@ const (
// config represents the configuration options available for the http.Handler
// and http.Transport types.
type config struct {
ServerName string
Tracer trace.Tracer
Meter metric.Meter
Propagators propagation.TextMapPropagator
@@ -64,7 +64,7 @@ func (o optionFunc) apply(c *config) {
func newConfig(opts ...Option) *config {
c := &config{
Propagators: otel.GetTextMapPropagator(),
MeterProvider: global.MeterProvider(),
MeterProvider: otel.GetMeterProvider(),
}
for _, opt := range opts {
opt.apply(c)
@@ -77,7 +77,7 @@ func newConfig(opts ...Option) *config {
c.Meter = c.MeterProvider.Meter(
instrumentationName,
metric.WithInstrumentationVersion(SemVersion()),
metric.WithInstrumentationVersion(Version()),
)
return c
@@ -198,3 +198,11 @@ func WithClientTrace(f func(context.Context) *httptrace.ClientTrace) Option {
c.ClientTrace = f
})
}
// WithServerName returns an Option that sets the name of the (virtual) server
// handling requests.
func WithServerName(server string) Option {
return optionFunc(func(c *config) {
c.ServerName = server
})
}

111
vendor/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp/handler.go generated vendored
View File

@@ -21,25 +21,19 @@ import (
"github.com/felixge/httpsnoop"
"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp/internal/semconvutil"
"go.opentelemetry.io/otel"
"go.opentelemetry.io/otel/attribute"
"go.opentelemetry.io/otel/metric"
"go.opentelemetry.io/otel/metric/instrument/syncfloat64"
"go.opentelemetry.io/otel/metric/instrument/syncint64"
"go.opentelemetry.io/otel/propagation"
semconv "go.opentelemetry.io/otel/semconv/v1.12.0"
semconv "go.opentelemetry.io/otel/semconv/v1.17.0"
"go.opentelemetry.io/otel/trace"
)
var _ http.Handler = &Handler{}
// Handler is http middleware that corresponds to the http.Handler interface and
// is designed to wrap a http.Mux (or equivalent), while individual routes on
// the mux are wrapped with WithRouteTag. A Handler will add various attributes
// to the span using the attribute.Keys defined in this package.
type Handler struct {
// middleware is an http middleware which wraps the next handler in a span.
type middleware struct {
operation string
handler http.Handler
server string
tracer trace.Tracer
meter metric.Meter
@@ -49,8 +43,8 @@ type Handler struct {
writeEvent bool
filters []Filter
spanNameFormatter func(string, *http.Request) string
counters map[string]syncint64.Counter
valueRecorders map[string]syncfloat64.Histogram
counters map[string]metric.Int64Counter
valueRecorders map[string]metric.Float64Histogram
publicEndpoint bool
publicEndpointFn func(*http.Request) bool
}
@@ -59,11 +53,17 @@ func defaultHandlerFormatter(operation string, _ *http.Request) string {
return operation
}
// NewHandler wraps the passed handler, functioning like middleware, in a span
// named after the operation and with any provided Options.
// NewHandler wraps the passed handler in a span named after the operation and
// enriches it with metrics.
func NewHandler(handler http.Handler, operation string, opts ...Option) http.Handler {
h := Handler{
handler: handler,
return NewMiddleware(operation, opts...)(handler)
}
// NewMiddleware returns a tracing and metrics instrumentation middleware.
// The handler returned by the middleware wraps a handler
// in a span named after the operation and enriches it with metrics.
func NewMiddleware(operation string, opts ...Option) func(http.Handler) http.Handler {
h := middleware{
operation: operation,
}
@@ -76,10 +76,14 @@ func NewHandler(handler http.Handler, operation string, opts ...Option) http.Han
h.configure(c)
h.createMeasures()
return &h
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
h.serveHTTP(w, r, next)
})
}
}
func (h *Handler) configure(c *config) {
func (h *middleware) configure(c *config) {
h.tracer = c.Tracer
h.meter = c.Meter
h.propagators = c.Propagators
@@ -90,6 +94,7 @@ func (h *Handler) configure(c *config) {
h.spanNameFormatter = c.SpanNameFormatter
h.publicEndpoint = c.PublicEndpoint
h.publicEndpointFn = c.PublicEndpointFn
h.server = c.ServerName
}
func handleErr(err error) {
@@ -98,17 +103,17 @@ func handleErr(err error) {
}
}
func (h *Handler) createMeasures() {
h.counters = make(map[string]syncint64.Counter)
h.valueRecorders = make(map[string]syncfloat64.Histogram)
func (h *middleware) createMeasures() {
h.counters = make(map[string]metric.Int64Counter)
h.valueRecorders = make(map[string]metric.Float64Histogram)
requestBytesCounter, err := h.meter.SyncInt64().Counter(RequestContentLength)
requestBytesCounter, err := h.meter.Int64Counter(RequestContentLength)
handleErr(err)
responseBytesCounter, err := h.meter.SyncInt64().Counter(ResponseContentLength)
responseBytesCounter, err := h.meter.Int64Counter(ResponseContentLength)
handleErr(err)
serverLatencyMeasure, err := h.meter.SyncFloat64().Histogram(ServerLatency)
serverLatencyMeasure, err := h.meter.Float64Histogram(ServerLatency)
handleErr(err)
h.counters[RequestContentLength] = requestBytesCounter
@@ -116,19 +121,27 @@ func (h *Handler) createMeasures() {
h.valueRecorders[ServerLatency] = serverLatencyMeasure
}
// ServeHTTP serves HTTP requests (http.Handler).
func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
// serveHTTP sets up tracing and calls the given next http.Handler with the span
// context injected into the request context.
func (h *middleware) serveHTTP(w http.ResponseWriter, r *http.Request, next http.Handler) {
requestStartTime := time.Now()
for _, f := range h.filters {
if !f(r) {
// Simply pass through to the handler if a filter rejects the request
h.handler.ServeHTTP(w, r)
next.ServeHTTP(w, r)
return
}
}
ctx := h.propagators.Extract(r.Context(), propagation.HeaderCarrier(r.Header))
opts := h.spanStartOptions
opts := []trace.SpanStartOption{
trace.WithAttributes(semconvutil.HTTPServerRequest(h.server, r)...),
}
if h.server != "" {
hostAttr := semconv.NetHostName(h.server)
opts = append(opts, trace.WithAttributes(hostAttr))
}
opts = append(opts, h.spanStartOptions...)
if h.publicEndpoint || (h.publicEndpointFn != nil && h.publicEndpointFn(r.WithContext(ctx))) {
opts = append(opts, trace.WithNewRoot())
// Linking incoming span context if any for public endpoint.
@@ -137,12 +150,6 @@ func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
}
}
opts = append([]trace.SpanStartOption{
trace.WithAttributes(semconv.NetAttributesFromHTTPRequest("tcp", r)...),
trace.WithAttributes(semconv.EndUserAttributesFromHTTPRequest(r)...),
trace.WithAttributes(semconv.HTTPServerAttributesFromHTTPRequest(h.operation, "", r)...),
}, opts...) // start with the configured options
tracer := h.tracer
if tracer == nil {
@@ -185,7 +192,7 @@ func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
record: writeRecordFunc,
ctx: ctx,
props: h.propagators,
statusCode: 200, // default status code in case the Handler doesn't write anything
statusCode: http.StatusOK, // default status code in case the Handler doesn't write anything
}
// Wrap w to use our ResponseWriter methods while also exposing
@@ -207,19 +214,23 @@ func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
labeler := &Labeler{}
ctx = injectLabeler(ctx, labeler)
h.handler.ServeHTTP(w, r.WithContext(ctx))
next.ServeHTTP(w, r.WithContext(ctx))
setAfterServeAttributes(span, bw.read, rww.written, rww.statusCode, bw.err, rww.err)
// Add metrics
attributes := append(labeler.Get(), semconv.HTTPServerMetricAttributesFromHTTPRequest(h.operation, r)...)
h.counters[RequestContentLength].Add(ctx, bw.read, attributes...)
h.counters[ResponseContentLength].Add(ctx, rww.written, attributes...)
attributes := append(labeler.Get(), semconvutil.HTTPServerRequestMetrics(h.server, r)...)
if rww.statusCode > 0 {
attributes = append(attributes, semconv.HTTPStatusCode(rww.statusCode))
}
o := metric.WithAttributes(attributes...)
h.counters[RequestContentLength].Add(ctx, bw.read, o)
h.counters[ResponseContentLength].Add(ctx, rww.written, o)
// Use floating point division here for higher precision (instead of Millisecond method).
elapsedTime := float64(time.Since(requestStartTime)) / float64(time.Millisecond)
h.valueRecorders[ServerLatency].Record(ctx, elapsedTime, attributes...)
h.valueRecorders[ServerLatency].Record(ctx, elapsedTime, o)
}
func setAfterServeAttributes(span trace.Span, read, wrote int64, statusCode int, rerr, werr error) {
@@ -236,8 +247,10 @@ func setAfterServeAttributes(span trace.Span, read, wrote int64, statusCode int,
if wrote > 0 {
attributes = append(attributes, WroteBytesKey.Int64(wrote))
}
attributes = append(attributes, semconv.HTTPAttributesFromHTTPStatusCode(statusCode)...)
span.SetStatus(semconv.SpanStatusFromHTTPStatusCodeAndSpanKind(statusCode, trace.SpanKindServer))
if statusCode > 0 {
attributes = append(attributes, semconv.HTTPStatusCode(statusCode))
}
span.SetStatus(semconvutil.HTTPServerStatus(statusCode))
if werr != nil && werr != io.EOF {
attributes = append(attributes, WriteErrorKey.String(werr.Error()))
@@ -245,12 +258,18 @@ func setAfterServeAttributes(span trace.Span, read, wrote int64, statusCode int,
span.SetAttributes(attributes...)
}
// WithRouteTag annotates a span with the provided route name using the
// RouteKey Tag.
// WithRouteTag annotates spans and metrics with the provided route name
// with HTTP route attribute.
func WithRouteTag(route string, h http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
attr := semconv.HTTPRouteKey.String(route)
span := trace.SpanFromContext(r.Context())
span.SetAttributes(semconv.HTTPRouteKey.String(route))
span.SetAttributes(attr)
labeler, _ := LabelerFromContext(r.Context())
labeler.Add(attr)
h.ServeHTTP(w, r)
})
}

21
vendor/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp/internal/semconvutil/gen.go generated vendored Normal file
View File

@@ -0,0 +1,21 @@
// Copyright The OpenTelemetry Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package semconvutil // import "go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp/internal/semconvutil"
// Generate semconvutil package:
//go:generate gotmpl --body=../../../../../../internal/shared/semconvutil/httpconv_test.go.tmpl "--data={}" --out=httpconv_test.go
//go:generate gotmpl --body=../../../../../../internal/shared/semconvutil/httpconv.go.tmpl "--data={}" --out=httpconv.go
//go:generate gotmpl --body=../../../../../../internal/shared/semconvutil/netconv_test.go.tmpl "--data={}" --out=netconv_test.go
//go:generate gotmpl --body=../../../../../../internal/shared/semconvutil/netconv.go.tmpl "--data={}" --out=netconv.go

552
vendor/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp/internal/semconvutil/httpconv.go generated vendored Normal file
View File

@@ -0,0 +1,552 @@
// Code created by gotmpl. DO NOT MODIFY.
// source: internal/shared/semconvutil/httpconv.go.tmpl
// Copyright The OpenTelemetry Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package semconvutil // import "go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp/internal/semconvutil"
import (
"fmt"
"net/http"
"strings"
"go.opentelemetry.io/otel/attribute"
"go.opentelemetry.io/otel/codes"
semconv "go.opentelemetry.io/otel/semconv/v1.17.0"
)
// HTTPClientResponse returns trace attributes for an HTTP response received by a
// client from a server. It will return the following attributes if the related
// values are defined in resp: "http.status.code",
// "http.response_content_length".
//
// This does not add all OpenTelemetry required attributes for an HTTP event,
// it assumes ClientRequest was used to create the span with a complete set of
// attributes. If a complete set of attributes can be generated using the
// request contained in resp. For example:
//
// append(HTTPClientResponse(resp), ClientRequest(resp.Request)...)
func HTTPClientResponse(resp *http.Response) []attribute.KeyValue {
return hc.ClientResponse(resp)
}
// HTTPClientRequest returns trace attributes for an HTTP request made by a client.
// The following attributes are always returned: "http.url", "http.flavor",
// "http.method", "net.peer.name". The following attributes are returned if the
// related values are defined in req: "net.peer.port", "http.user_agent",
// "http.request_content_length", "enduser.id".
func HTTPClientRequest(req *http.Request) []attribute.KeyValue {
return hc.ClientRequest(req)
}
// HTTPClientStatus returns a span status code and message for an HTTP status code
// value received by a client.
func HTTPClientStatus(code int) (codes.Code, string) {
return hc.ClientStatus(code)
}
// HTTPServerRequest returns trace attributes for an HTTP request received by a
// server.
//
// The server must be the primary server name if it is known. For example this
// would be the ServerName directive
// (https://httpd.apache.org/docs/2.4/mod/core.html#servername) for an Apache
// server, and the server_name directive
// (http://nginx.org/en/docs/http/ngx_http_core_module.html#server_name) for an
// nginx server. More generically, the primary server name would be the host
// header value that matches the default virtual host of an HTTP server. It
// should include the host identifier and if a port is used to route to the
// server that port identifier should be included as an appropriate port
// suffix.
//
// If the primary server name is not known, server should be an empty string.
// The req Host will be used to determine the server instead.
//
// The following attributes are always returned: "http.method", "http.scheme",
// "http.flavor", "http.target", "net.host.name". The following attributes are
// returned if they related values are defined in req: "net.host.port",
// "net.sock.peer.addr", "net.sock.peer.port", "http.user_agent", "enduser.id",
// "http.client_ip".
func HTTPServerRequest(server string, req *http.Request) []attribute.KeyValue {
return hc.ServerRequest(server, req)
}
// HTTPServerRequestMetrics returns metric attributes for an HTTP request received by a
// server.
//
// The server must be the primary server name if it is known. For example this
// would be the ServerName directive
// (https://httpd.apache.org/docs/2.4/mod/core.html#servername) for an Apache
// server, and the server_name directive
// (http://nginx.org/en/docs/http/ngx_http_core_module.html#server_name) for an
// nginx server. More generically, the primary server name would be the host
// header value that matches the default virtual host of an HTTP server. It
// should include the host identifier and if a port is used to route to the
// server that port identifier should be included as an appropriate port
// suffix.
//
// If the primary server name is not known, server should be an empty string.
// The req Host will be used to determine the server instead.
//
// The following attributes are always returned: "http.method", "http.scheme",
// "http.flavor", "net.host.name". The following attributes are
// returned if they related values are defined in req: "net.host.port".
func HTTPServerRequestMetrics(server string, req *http.Request) []attribute.KeyValue {
return hc.ServerRequestMetrics(server, req)
}
// HTTPServerStatus returns a span status code and message for an HTTP status code
// value returned by a server. Status codes in the 400-499 range are not
// returned as errors.
func HTTPServerStatus(code int) (codes.Code, string) {
return hc.ServerStatus(code)
}
// HTTPRequestHeader returns the contents of h as attributes.
//
// Instrumentation should require an explicit configuration of which headers to
// captured and then prune what they pass here. Including all headers can be a
// security risk - explicit configuration helps avoid leaking sensitive
// information.
//
// The User-Agent header is already captured in the http.user_agent attribute
// from ClientRequest and ServerRequest. Instrumentation may provide an option
// to capture that header here even though it is not recommended. Otherwise,
// instrumentation should filter that out of what is passed.
func HTTPRequestHeader(h http.Header) []attribute.KeyValue {
return hc.RequestHeader(h)
}
// HTTPResponseHeader returns the contents of h as attributes.
//
// Instrumentation should require an explicit configuration of which headers to
// captured and then prune what they pass here. Including all headers can be a
// security risk - explicit configuration helps avoid leaking sensitive
// information.
//
// The User-Agent header is already captured in the http.user_agent attribute
// from ClientRequest and ServerRequest. Instrumentation may provide an option
// to capture that header here even though it is not recommended. Otherwise,
// instrumentation should filter that out of what is passed.
func HTTPResponseHeader(h http.Header) []attribute.KeyValue {
return hc.ResponseHeader(h)
}
// httpConv are the HTTP semantic convention attributes defined for a version
// of the OpenTelemetry specification.
type httpConv struct {
NetConv *netConv
EnduserIDKey attribute.Key
HTTPClientIPKey attribute.Key
HTTPFlavorKey attribute.Key
HTTPMethodKey attribute.Key
HTTPRequestContentLengthKey attribute.Key
HTTPResponseContentLengthKey attribute.Key
HTTPRouteKey attribute.Key
HTTPSchemeHTTP attribute.KeyValue
HTTPSchemeHTTPS attribute.KeyValue
HTTPStatusCodeKey attribute.Key
HTTPTargetKey attribute.Key
HTTPURLKey attribute.Key
HTTPUserAgentKey attribute.Key
}
var hc = &httpConv{
NetConv: nc,
EnduserIDKey: semconv.EnduserIDKey,
HTTPClientIPKey: semconv.HTTPClientIPKey,
HTTPFlavorKey: semconv.HTTPFlavorKey,
HTTPMethodKey: semconv.HTTPMethodKey,
HTTPRequestContentLengthKey: semconv.HTTPRequestContentLengthKey,
HTTPResponseContentLengthKey: semconv.HTTPResponseContentLengthKey,
HTTPRouteKey: semconv.HTTPRouteKey,
HTTPSchemeHTTP: semconv.HTTPSchemeHTTP,
HTTPSchemeHTTPS: semconv.HTTPSchemeHTTPS,
HTTPStatusCodeKey: semconv.HTTPStatusCodeKey,
HTTPTargetKey: semconv.HTTPTargetKey,
HTTPURLKey: semconv.HTTPURLKey,
HTTPUserAgentKey: semconv.HTTPUserAgentKey,
}
// ClientResponse returns attributes for an HTTP response received by a client
// from a server. The following attributes are returned if the related values
// are defined in resp: "http.status.code", "http.response_content_length".
//
// This does not add all OpenTelemetry required attributes for an HTTP event,
// it assumes ClientRequest was used to create the span with a complete set of
// attributes. If a complete set of attributes can be generated using the
// request contained in resp. For example:
//
// append(ClientResponse(resp), ClientRequest(resp.Request)...)
func (c *httpConv) ClientResponse(resp *http.Response) []attribute.KeyValue {
var n int
if resp.StatusCode > 0 {
n++
}
if resp.ContentLength > 0 {
n++
}
attrs := make([]attribute.KeyValue, 0, n)
if resp.StatusCode > 0 {
attrs = append(attrs, c.HTTPStatusCodeKey.Int(resp.StatusCode))
}
if resp.ContentLength > 0 {
attrs = append(attrs, c.HTTPResponseContentLengthKey.Int(int(resp.ContentLength)))
}
return attrs
}
// ClientRequest returns attributes for an HTTP request made by a client. The
// following attributes are always returned: "http.url", "http.flavor",
// "http.method", "net.peer.name". The following attributes are returned if the
// related values are defined in req: "net.peer.port", "http.user_agent",
// "http.request_content_length", "enduser.id".
func (c *httpConv) ClientRequest(req *http.Request) []attribute.KeyValue {
n := 3 // URL, peer name, proto, and method.
var h string
if req.URL != nil {
h = req.URL.Host
}
peer, p := firstHostPort(h, req.Header.Get("Host"))
port := requiredHTTPPort(req.URL != nil && req.URL.Scheme == "https", p)
if port > 0 {
n++
}
useragent := req.UserAgent()
if useragent != "" {
n++
}
if req.ContentLength > 0 {
n++
}
userID, _, hasUserID := req.BasicAuth()
if hasUserID {
n++
}
attrs := make([]attribute.KeyValue, 0, n)
attrs = append(attrs, c.method(req.Method))
attrs = append(attrs, c.flavor(req.Proto))
var u string
if req.URL != nil {
// Remove any username/password info that may be in the URL.
userinfo := req.URL.User
req.URL.User = nil
u = req.URL.String()
// Restore any username/password info that was removed.
req.URL.User = userinfo
}
attrs = append(attrs, c.HTTPURLKey.String(u))
attrs = append(attrs, c.NetConv.PeerName(peer))
if port > 0 {
attrs = append(attrs, c.NetConv.PeerPort(port))
}
if useragent != "" {
attrs = append(attrs, c.HTTPUserAgentKey.String(useragent))
}
if l := req.ContentLength; l > 0 {
attrs = append(attrs, c.HTTPRequestContentLengthKey.Int64(l))
}
if hasUserID {
attrs = append(attrs, c.EnduserIDKey.String(userID))
}
return attrs
}
// ServerRequest returns attributes for an HTTP request received by a server.
//
// The server must be the primary server name if it is known. For example this
// would be the ServerName directive
// (https://httpd.apache.org/docs/2.4/mod/core.html#servername) for an Apache
// server, and the server_name directive
// (http://nginx.org/en/docs/http/ngx_http_core_module.html#server_name) for an
// nginx server. More generically, the primary server name would be the host
// header value that matches the default virtual host of an HTTP server. It
// should include the host identifier and if a port is used to route to the
// server that port identifier should be included as an appropriate port
// suffix.
//
// If the primary server name is not known, server should be an empty string.
// The req Host will be used to determine the server instead.
//
// The following attributes are always returned: "http.method", "http.scheme",
// "http.flavor", "http.target", "net.host.name". The following attributes are
// returned if they related values are defined in req: "net.host.port",
// "net.sock.peer.addr", "net.sock.peer.port", "http.user_agent", "enduser.id",
// "http.client_ip".
func (c *httpConv) ServerRequest(server string, req *http.Request) []attribute.KeyValue {
// TODO: This currently does not add the specification required
// `http.target` attribute. It has too high of a cardinality to safely be
// added. An alternate should be added, or this comment removed, when it is
// addressed by the specification. If it is ultimately decided to continue
// not including the attribute, the HTTPTargetKey field of the httpConv
// should be removed as well.
n := 4 // Method, scheme, proto, and host name.
var host string
var p int
if server == "" {
host, p = splitHostPort(req.Host)
} else {
// Prioritize the primary server name.
host, p = splitHostPort(server)
if p < 0 {
_, p = splitHostPort(req.Host)
}
}
hostPort := requiredHTTPPort(req.TLS != nil, p)
if hostPort > 0 {
n++
}
peer, peerPort := splitHostPort(req.RemoteAddr)
if peer != "" {
n++
if peerPort > 0 {
n++
}
}
useragent := req.UserAgent()
if useragent != "" {
n++
}
userID, _, hasUserID := req.BasicAuth()
if hasUserID {
n++
}
clientIP := serverClientIP(req.Header.Get("X-Forwarded-For"))
if clientIP != "" {
n++
}
attrs := make([]attribute.KeyValue, 0, n)
attrs = append(attrs, c.method(req.Method))
attrs = append(attrs, c.scheme(req.TLS != nil))
attrs = append(attrs, c.flavor(req.Proto))
attrs = append(attrs, c.NetConv.HostName(host))
if hostPort > 0 {
attrs = append(attrs, c.NetConv.HostPort(hostPort))
}
if peer != "" {
// The Go HTTP server sets RemoteAddr to "IP:port", this will not be a
// file-path that would be interpreted with a sock family.
attrs = append(attrs, c.NetConv.SockPeerAddr(peer))
if peerPort > 0 {
attrs = append(attrs, c.NetConv.SockPeerPort(peerPort))
}
}
if useragent != "" {
attrs = append(attrs, c.HTTPUserAgentKey.String(useragent))
}
if hasUserID {
attrs = append(attrs, c.EnduserIDKey.String(userID))
}
if clientIP != "" {
attrs = append(attrs, c.HTTPClientIPKey.String(clientIP))
}
return attrs
}
// ServerRequestMetrics returns metric attributes for an HTTP request received
// by a server.
//
// The server must be the primary server name if it is known. For example this
// would be the ServerName directive
// (https://httpd.apache.org/docs/2.4/mod/core.html#servername) for an Apache
// server, and the server_name directive
// (http://nginx.org/en/docs/http/ngx_http_core_module.html#server_name) for an
// nginx server. More generically, the primary server name would be the host
// header value that matches the default virtual host of an HTTP server. It
// should include the host identifier and if a port is used to route to the
// server that port identifier should be included as an appropriate port
// suffix.
//
// If the primary server name is not known, server should be an empty string.
// The req Host will be used to determine the server instead.
//
// The following attributes are always returned: "http.method", "http.scheme",
// "http.flavor", "net.host.name". The following attributes are
// returned if they related values are defined in req: "net.host.port".
func (c *httpConv) ServerRequestMetrics(server string, req *http.Request) []attribute.KeyValue {
// TODO: This currently does not add the specification required
// `http.target` attribute. It has too high of a cardinality to safely be
// added. An alternate should be added, or this comment removed, when it is
// addressed by the specification. If it is ultimately decided to continue
// not including the attribute, the HTTPTargetKey field of the httpConv
// should be removed as well.
n := 4 // Method, scheme, proto, and host name.
var host string
var p int
if server == "" {
host, p = splitHostPort(req.Host)
} else {
// Prioritize the primary server name.
host, p = splitHostPort(server)
if p < 0 {
_, p = splitHostPort(req.Host)
}
}
hostPort := requiredHTTPPort(req.TLS != nil, p)
if hostPort > 0 {
n++
}
attrs := make([]attribute.KeyValue, 0, n)
attrs = append(attrs, c.methodMetric(req.Method))
attrs = append(attrs, c.scheme(req.TLS != nil))
attrs = append(attrs, c.flavor(req.Proto))
attrs = append(attrs, c.NetConv.HostName(host))
if hostPort > 0 {
attrs = append(attrs, c.NetConv.HostPort(hostPort))
}
return attrs
}
func (c *httpConv) method(method string) attribute.KeyValue {
if method == "" {
return c.HTTPMethodKey.String(http.MethodGet)
}
return c.HTTPMethodKey.String(method)
}
func (c *httpConv) methodMetric(method string) attribute.KeyValue {
method = strings.ToUpper(method)
switch method {
case http.MethodConnect, http.MethodDelete, http.MethodGet, http.MethodHead, http.MethodOptions, http.MethodPatch, http.MethodPost, http.MethodPut, http.MethodTrace:
default:
method = "_OTHER"
}
return c.HTTPMethodKey.String(method)
}
func (c *httpConv) scheme(https bool) attribute.KeyValue { // nolint:revive
if https {
return c.HTTPSchemeHTTPS
}
return c.HTTPSchemeHTTP
}
func (c *httpConv) flavor(proto string) attribute.KeyValue {
switch proto {
case "HTTP/1.0":
return c.HTTPFlavorKey.String("1.0")
case "HTTP/1.1":
return c.HTTPFlavorKey.String("1.1")
case "HTTP/2":
return c.HTTPFlavorKey.String("2.0")
case "HTTP/3":
return c.HTTPFlavorKey.String("3.0")
default:
return c.HTTPFlavorKey.String(proto)
}
}
func serverClientIP(xForwardedFor string) string {
if idx := strings.Index(xForwardedFor, ","); idx >= 0 {
xForwardedFor = xForwardedFor[:idx]
}
return xForwardedFor
}
func requiredHTTPPort(https bool, port int) int { // nolint:revive
if https {
if port > 0 && port != 443 {
return port
}
} else {
if port > 0 && port != 80 {
return port
}
}
return -1
}
// Return the request host and port from the first non-empty source.
func firstHostPort(source ...string) (host string, port int) {
for _, hostport := range source {
host, port = splitHostPort(hostport)
if host != "" || port > 0 {
break
}
}
return
}
// RequestHeader returns the contents of h as OpenTelemetry attributes.
func (c *httpConv) RequestHeader(h http.Header) []attribute.KeyValue {
return c.header("http.request.header", h)
}
// ResponseHeader returns the contents of h as OpenTelemetry attributes.
func (c *httpConv) ResponseHeader(h http.Header) []attribute.KeyValue {
return c.header("http.response.header", h)
}
func (c *httpConv) header(prefix string, h http.Header) []attribute.KeyValue {
key := func(k string) attribute.Key {
k = strings.ToLower(k)
k = strings.ReplaceAll(k, "-", "_")
k = fmt.Sprintf("%s.%s", prefix, k)
return attribute.Key(k)
}
attrs := make([]attribute.KeyValue, 0, len(h))
for k, v := range h {
attrs = append(attrs, key(k).StringSlice(v))
}
return attrs
}
// ClientStatus returns a span status code and message for an HTTP status code
// value received by a client.
func (c *httpConv) ClientStatus(code int) (codes.Code, string) {
if code < 100 || code >= 600 {
return codes.Error, fmt.Sprintf("Invalid HTTP status code %d", code)
}
if code >= 400 {
return codes.Error, ""
}
return codes.Unset, ""
}
// ServerStatus returns a span status code and message for an HTTP status code
// value returned by a server. Status codes in the 400-499 range are not
// returned as errors.
func (c *httpConv) ServerStatus(code int) (codes.Code, string) {
if code < 100 || code >= 600 {
return codes.Error, fmt.Sprintf("Invalid HTTP status code %d", code)
}
if code >= 500 {
return codes.Error, ""
}
return codes.Unset, ""
}

368
vendor/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp/internal/semconvutil/netconv.go generated vendored Normal file
View File

@@ -0,0 +1,368 @@
// Code created by gotmpl. DO NOT MODIFY.
// source: internal/shared/semconvutil/netconv.go.tmpl
// Copyright The OpenTelemetry Authors
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package semconvutil // import "go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp/internal/semconvutil"
import (
"net"
"strconv"
"strings"
"go.opentelemetry.io/otel/attribute"
semconv "go.opentelemetry.io/otel/semconv/v1.17.0"
)
// NetTransport returns a trace attribute describing the transport protocol of the
// passed network. See the net.Dial for information about acceptable network
// values.
func NetTransport(network string) attribute.KeyValue {
return nc.Transport(network)
}
// NetClient returns trace attributes for a client network connection to address.
// See net.Dial for information about acceptable address values, address should
// be the same as the one used to create conn. If conn is nil, only network
// peer attributes will be returned that describe address. Otherwise, the
// socket level information about conn will also be included.
func NetClient(address string, conn net.Conn) []attribute.KeyValue {
return nc.Client(address, conn)
}
// NetServer returns trace attributes for a network listener listening at address.
// See net.Listen for information about acceptable address values, address
// should be the same as the one used to create ln. If ln is nil, only network
// host attributes will be returned that describe address. Otherwise, the
// socket level information about ln will also be included.
func NetServer(address string, ln net.Listener) []attribute.KeyValue {
return nc.Server(address, ln)
}
// netConv are the network semantic convention attributes defined for a version
// of the OpenTelemetry specification.
type netConv struct {
NetHostNameKey attribute.Key
NetHostPortKey attribute.Key
NetPeerNameKey attribute.Key
NetPeerPortKey attribute.Key
NetSockFamilyKey attribute.Key
NetSockPeerAddrKey attribute.Key
NetSockPeerPortKey attribute.Key
NetSockHostAddrKey attribute.Key
NetSockHostPortKey attribute.Key
NetTransportOther attribute.KeyValue
NetTransportTCP attribute.KeyValue
NetTransportUDP attribute.KeyValue
NetTransportInProc attribute.KeyValue
}
var nc = &netConv{
NetHostNameKey: semconv.NetHostNameKey,
NetHostPortKey: semconv.NetHostPortKey,
NetPeerNameKey: semconv.NetPeerNameKey,
NetPeerPortKey: semconv.NetPeerPortKey,
NetSockFamilyKey: semconv.NetSockFamilyKey,
NetSockPeerAddrKey: semconv.NetSockPeerAddrKey,
NetSockPeerPortKey: semconv.NetSockPeerPortKey,
NetSockHostAddrKey: semconv.NetSockHostAddrKey,
NetSockHostPortKey: semconv.NetSockHostPortKey,
NetTransportOther: semconv.NetTransportOther,
NetTransportTCP: semconv.NetTransportTCP,
NetTransportUDP: semconv.NetTransportUDP,
NetTransportInProc: semconv.NetTransportInProc,
}
func (c *netConv) Transport(network string) attribute.KeyValue {
switch network {
case "tcp", "tcp4", "tcp6":
return c.NetTransportTCP
case "udp", "udp4", "udp6":
return c.NetTransportUDP
case "unix", "unixgram", "unixpacket":
return c.NetTransportInProc
default:
// "ip:*", "ip4:*", and "ip6:*" all are considered other.
return c.NetTransportOther
}
}
// Host returns attributes for a network host address.
func (c *netConv) Host(address string) []attribute.KeyValue {
h, p := splitHostPort(address)
var n int
if h != "" {
n++
if p > 0 {
n++
}
}
if n == 0 {
return nil
}
attrs := make([]attribute.KeyValue, 0, n)
attrs = append(attrs, c.HostName(h))
if p > 0 {
attrs = append(attrs, c.HostPort(int(p)))
}
return attrs
}
// Server returns attributes for a network listener listening at address. See
// net.Listen for information about acceptable address values, address should
// be the same as the one used to create ln. If ln is nil, only network host
// attributes will be returned that describe address. Otherwise, the socket
// level information about ln will also be included.
func (c *netConv) Server(address string, ln net.Listener) []attribute.KeyValue {
if ln == nil {
return c.Host(address)
}
lAddr := ln.Addr()
if lAddr == nil {
return c.Host(address)
}
hostName, hostPort := splitHostPort(address)
sockHostAddr, sockHostPort := splitHostPort(lAddr.String())
network := lAddr.Network()
sockFamily := family(network, sockHostAddr)
n := nonZeroStr(hostName, network, sockHostAddr, sockFamily)
n += positiveInt(hostPort, sockHostPort)
attr := make([]attribute.KeyValue, 0, n)
if hostName != "" {
attr = append(attr, c.HostName(hostName))
if hostPort > 0 {
// Only if net.host.name is set should net.host.port be.
attr = append(attr, c.HostPort(hostPort))
}
}
if network != "" {
attr = append(attr, c.Transport(network))
}
if sockFamily != "" {
attr = append(attr, c.NetSockFamilyKey.String(sockFamily))
}
if sockHostAddr != "" {
attr = append(attr, c.NetSockHostAddrKey.String(sockHostAddr))
if sockHostPort > 0 {
// Only if net.sock.host.addr is set should net.sock.host.port be.
attr = append(attr, c.NetSockHostPortKey.Int(sockHostPort))
}
}
return attr
}
func (c *netConv) HostName(name string) attribute.KeyValue {
return c.NetHostNameKey.String(name)
}
func (c *netConv) HostPort(port int) attribute.KeyValue {
return c.NetHostPortKey.Int(port)
}
// Client returns attributes for a client network connection to address. See
// net.Dial for information about acceptable address values, address should be
// the same as the one used to create conn. If conn is nil, only network peer
// attributes will be returned that describe address. Otherwise, the socket
// level information about conn will also be included.
func (c *netConv) Client(address string, conn net.Conn) []attribute.KeyValue {
if conn == nil {
return c.Peer(address)
}
lAddr, rAddr := conn.LocalAddr(), conn.RemoteAddr()
var network string
switch {
case lAddr != nil:
network = lAddr.Network()
case rAddr != nil:
network = rAddr.Network()
default:
return c.Peer(address)
}
peerName, peerPort := splitHostPort(address)
var (
sockFamily string
sockPeerAddr string
sockPeerPort int
sockHostAddr string
sockHostPort int
)
if lAddr != nil {
sockHostAddr, sockHostPort = splitHostPort(lAddr.String())
}
if rAddr != nil {
sockPeerAddr, sockPeerPort = splitHostPort(rAddr.String())
}
switch {
case sockHostAddr != "":
sockFamily = family(network, sockHostAddr)
case sockPeerAddr != "":
sockFamily = family(network, sockPeerAddr)
}
n := nonZeroStr(peerName, network, sockPeerAddr, sockHostAddr, sockFamily)
n += positiveInt(peerPort, sockPeerPort, sockHostPort)
attr := make([]attribute.KeyValue, 0, n)
if peerName != "" {
attr = append(attr, c.PeerName(peerName))
if peerPort > 0 {
// Only if net.peer.name is set should net.peer.port be.
attr = append(attr, c.PeerPort(peerPort))
}
}
if network != "" {
attr = append(attr, c.Transport(network))
}
if sockFamily != "" {
attr = append(attr, c.NetSockFamilyKey.String(sockFamily))
}
if sockPeerAddr != "" {
attr = append(attr, c.NetSockPeerAddrKey.String(sockPeerAddr))
if sockPeerPort > 0 {
// Only if net.sock.peer.addr is set should net.sock.peer.port be.
attr = append(attr, c.NetSockPeerPortKey.Int(sockPeerPort))
}
}
if sockHostAddr != "" {
attr = append(attr, c.NetSockHostAddrKey.String(sockHostAddr))
if sockHostPort > 0 {
// Only if net.sock.host.addr is set should net.sock.host.port be.
attr = append(attr, c.NetSockHostPortKey.Int(sockHostPort))
}
}
return attr
}
func family(network, address string) string {
switch network {
case "unix", "unixgram", "unixpacket":
return "unix"
default:
if ip := net.ParseIP(address); ip != nil {
if ip.To4() == nil {
return "inet6"
}
return "inet"
}
}
return ""
}
func nonZeroStr(strs ...string) int {
var n int
for _, str := range strs {
if str != "" {
n++
}
}
return n
}
func positiveInt(ints ...int) int {
var n int
for _, i := range ints {
if i > 0 {
n++
}
}
return n
}
// Peer returns attributes for a network peer address.
func (c *netConv) Peer(address string) []attribute.KeyValue {
h, p := splitHostPort(address)
var n int
if h != "" {
n++
if p > 0 {
n++
}
}
if n == 0 {
return nil
}
attrs := make([]attribute.KeyValue, 0, n)
attrs = append(attrs, c.PeerName(h))
if p > 0 {
attrs = append(attrs, c.PeerPort(int(p)))
}
return attrs
}
func (c *netConv) PeerName(name string) attribute.KeyValue {
return c.NetPeerNameKey.String(name)
}
func (c *netConv) PeerPort(port int) attribute.KeyValue {
return c.NetPeerPortKey.Int(port)
}
func (c *netConv) SockPeerAddr(addr string) attribute.KeyValue {
return c.NetSockPeerAddrKey.String(addr)
}
func (c *netConv) SockPeerPort(port int) attribute.KeyValue {
return c.NetSockPeerPortKey.Int(port)
}
// splitHostPort splits a network address hostport of the form "host",
// "host%zone", "[host]", "[host%zone], "host:port", "host%zone:port",
// "[host]:port", "[host%zone]:port", or ":port" into host or host%zone and
// port.
//
// An empty host is returned if it is not provided or unparsable. A negative
// port is returned if it is not provided or unparsable.
func splitHostPort(hostport string) (host string, port int) {
port = -1
if strings.HasPrefix(hostport, "[") {
addrEnd := strings.LastIndex(hostport, "]")
if addrEnd < 0 {
// Invalid hostport.
return
}
if i := strings.LastIndex(hostport[addrEnd:], ":"); i < 0 {
host = hostport[1:addrEnd]
return
}
} else {
if i := strings.LastIndex(hostport, ":"); i < 0 {
host = hostport
return
}
}
host, pStr, err := net.SplitHostPort(hostport)
if err != nil {
return
}
p, err := strconv.ParseUint(pStr, 10, 16)
if err != nil {
return
}
return host, int(p)
}

10
vendor/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp/transport.go generated vendored
View File

@@ -20,10 +20,10 @@ import (
"net/http"
"net/http/httptrace"
"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp/internal/semconvutil"
"go.opentelemetry.io/otel"
"go.opentelemetry.io/otel/codes"
"go.opentelemetry.io/otel/propagation"
semconv "go.opentelemetry.io/otel/semconv/v1.12.0"
"go.opentelemetry.io/otel/trace"
)
@@ -109,8 +109,8 @@ func (t *Transport) RoundTrip(r *http.Request) (*http.Response, error) {
ctx = httptrace.WithClientTrace(ctx, t.clientTrace(ctx))
}
r = r.WithContext(ctx)
span.SetAttributes(semconv.HTTPClientAttributesFromHTTPRequest(r)...)
r = r.Clone(ctx) // According to RoundTripper spec, we shouldn't modify the origin request.
span.SetAttributes(semconvutil.HTTPClientRequest(r)...)
t.propagators.Inject(ctx, propagation.HeaderCarrier(r.Header))
res, err := t.rt.RoundTrip(r)
@@ -121,8 +121,8 @@ func (t *Transport) RoundTrip(r *http.Request) (*http.Response, error) {
return res, err
}
span.SetAttributes(semconv.HTTPAttributesFromHTTPStatusCode(res.StatusCode)...)
span.SetStatus(semconv.SpanStatusFromHTTPStatusCode(res.StatusCode))
span.SetAttributes(semconvutil.HTTPClientResponse(res)...)
span.SetStatus(semconvutil.HTTPClientStatus(res.StatusCode))
res.Body = newWrappedBody(span, res.Body)
return res, err

6
vendor/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp/version.go generated vendored
View File

@@ -16,11 +16,13 @@ package otelhttp // import "go.opentelemetry.io/contrib/instrumentation/net/http
// Version is the current release version of the otelhttp instrumentation.
func Version() string {
return "0.37.0"
return "0.45.0"
// This string is updated by the pre_release.sh script during release
}
// SemVersion is the semantic version to be supplied to tracer/meter creation.
//
// Deprecated: Use [Version] instead.
func SemVersion() string {
return "semver:" + Version()
return Version()
}

14
vendor/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp/wrap.go generated vendored
View File

@@ -50,7 +50,7 @@ func (w *bodyWrapper) Close() error {
var _ http.ResponseWriter = &respWriterWrapper{}
// respWriterWrapper wraps a http.ResponseWriter in order to track the number of
// bytes written, the last error, and to catch the returned statusCode
// bytes written, the last error, and to catch the first written statusCode.
// TODO: The wrapped http.ResponseWriter doesn't implement any of the optional
// types (http.Hijacker, http.Pusher, http.CloseNotifier, http.Flusher, etc)
// that may be useful when using it in real life situations.
@@ -85,11 +85,15 @@ func (w *respWriterWrapper) Write(p []byte) (int, error) {
return n, err
}
// WriteHeader persists initial statusCode for span attribution.
// All calls to WriteHeader will be propagated to the underlying ResponseWriter
// and will persist the statusCode from the first call.
// Blocking consecutive calls to WriteHeader alters expected behavior and will
// remove warning logs from net/http where developers will notice incorrect handler implementations.
func (w *respWriterWrapper) WriteHeader(statusCode int) {
if w.wroteHeader {
return
if !w.wroteHeader {
w.wroteHeader = true
w.statusCode = statusCode
}
w.wroteHeader = true
w.statusCode = statusCode
w.ResponseWriter.WriteHeader(statusCode)
}

Some files were not shown because too many files have changed in this diff Show More