Add Conda package registry (#22262)
This PR adds a [Conda](https://conda.io/) package registry.
This commit is contained in:
parent
5882e179a9
commit
6ba9ff7b48
|
@ -2458,6 +2458,8 @@ ROUTER = console
|
||||||
;LIMIT_SIZE_COMPOSER = -1
|
;LIMIT_SIZE_COMPOSER = -1
|
||||||
;; Maximum size of a Conan upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
|
;; Maximum size of a Conan upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
|
||||||
;LIMIT_SIZE_CONAN = -1
|
;LIMIT_SIZE_CONAN = -1
|
||||||
|
;; Maximum size of a Conda upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
|
||||||
|
;LIMIT_SIZE_CONDA = -1
|
||||||
;; Maximum size of a Container upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
|
;; Maximum size of a Container upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
|
||||||
;LIMIT_SIZE_CONTAINER = -1
|
;LIMIT_SIZE_CONTAINER = -1
|
||||||
;; Maximum size of a Generic upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
|
;; Maximum size of a Generic upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
|
||||||
|
|
|
@ -1214,6 +1214,7 @@ Task queue configuration has been moved to `queue.task`. However, the below conf
|
||||||
- `LIMIT_TOTAL_OWNER_SIZE`: **-1**: Maximum size of packages a single owner can use (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
|
- `LIMIT_TOTAL_OWNER_SIZE`: **-1**: Maximum size of packages a single owner can use (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
|
||||||
- `LIMIT_SIZE_COMPOSER`: **-1**: Maximum size of a Composer upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
|
- `LIMIT_SIZE_COMPOSER`: **-1**: Maximum size of a Composer upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
|
||||||
- `LIMIT_SIZE_CONAN`: **-1**: Maximum size of a Conan upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
|
- `LIMIT_SIZE_CONAN`: **-1**: Maximum size of a Conan upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
|
||||||
|
- `LIMIT_SIZE_CONDA`: **-1**: Maximum size of a Conda upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
|
||||||
- `LIMIT_SIZE_CONTAINER`: **-1**: Maximum size of a Container upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
|
- `LIMIT_SIZE_CONTAINER`: **-1**: Maximum size of a Container upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
|
||||||
- `LIMIT_SIZE_GENERIC`: **-1**: Maximum size of a Generic upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
|
- `LIMIT_SIZE_GENERIC`: **-1**: Maximum size of a Generic upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
|
||||||
- `LIMIT_SIZE_HELM`: **-1**: Maximum size of a Helm upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
|
- `LIMIT_SIZE_HELM`: **-1**: Maximum size of a Helm upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
|
||||||
|
|
85
docs/content/doc/packages/conda.en-us.md
Normal file
85
docs/content/doc/packages/conda.en-us.md
Normal file
|
@ -0,0 +1,85 @@
|
||||||
|
---
|
||||||
|
date: "2022-12-28T00:00:00+00:00"
|
||||||
|
title: "Conda Packages Repository"
|
||||||
|
slug: "packages/conda"
|
||||||
|
draft: false
|
||||||
|
toc: false
|
||||||
|
menu:
|
||||||
|
sidebar:
|
||||||
|
parent: "packages"
|
||||||
|
name: "Conda"
|
||||||
|
weight: 25
|
||||||
|
identifier: "conda"
|
||||||
|
---
|
||||||
|
|
||||||
|
# Conda Packages Repository
|
||||||
|
|
||||||
|
Publish [Conda](https://docs.conda.io/en/latest/) packages for your user or organization.
|
||||||
|
|
||||||
|
**Table of Contents**
|
||||||
|
|
||||||
|
{{< toc >}}
|
||||||
|
|
||||||
|
## Requirements
|
||||||
|
|
||||||
|
To work with the Conda package registry, you need to use [conda](https://docs.conda.io/projects/conda/en/stable/user-guide/install/index.html).
|
||||||
|
|
||||||
|
## Configuring the package registry
|
||||||
|
|
||||||
|
To register the package registry and provide credentials, edit your `.condarc` file:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
channel_alias: https://gitea.example.com/api/packages/{owner}/conda
|
||||||
|
channels:
|
||||||
|
- https://gitea.example.com/api/packages/{owner}/conda
|
||||||
|
default_channels:
|
||||||
|
- https://gitea.example.com/api/packages/{owner}/conda
|
||||||
|
```
|
||||||
|
|
||||||
|
| Placeholder | Description |
|
||||||
|
| ------------ | ----------- |
|
||||||
|
| `owner` | The owner of the package. |
|
||||||
|
|
||||||
|
See the [official documentation](https://conda.io/projects/conda/en/latest/user-guide/configuration/use-condarc.html) for explanations of the individual settings.
|
||||||
|
|
||||||
|
If you need to provide credentials, you may embed them as part of the channel url (`https://user:password@gitea.example.com/...`).
|
||||||
|
|
||||||
|
## Publish a package
|
||||||
|
|
||||||
|
To publish a package, perform a HTTP PUT operation with the package content in the request body.
|
||||||
|
|
||||||
|
```
|
||||||
|
PUT https://gitea.example.com/api/packages/{owner}/conda/{channel}/{filename}
|
||||||
|
```
|
||||||
|
|
||||||
|
| Placeholder | Description |
|
||||||
|
| ------------ | ----------- |
|
||||||
|
| `owner` | The owner of the package. |
|
||||||
|
| `channel` | The [channel](https://conda.io/projects/conda/en/latest/user-guide/concepts/channels.html) of the package. (optional) |
|
||||||
|
| `filename` | The name of the file. |
|
||||||
|
|
||||||
|
Example request using HTTP Basic authentication:
|
||||||
|
|
||||||
|
```shell
|
||||||
|
curl --user your_username:your_password_or_token \
|
||||||
|
--upload-file path/to/package-1.0.conda \
|
||||||
|
https://gitea.example.com/api/packages/testuser/conda/package-1.0.conda
|
||||||
|
```
|
||||||
|
|
||||||
|
You cannot publish a package if a package of the same name and version already exists. You must delete the existing package first.
|
||||||
|
|
||||||
|
## Install a package
|
||||||
|
|
||||||
|
To install a package from the package registry, execute one of the following commands:
|
||||||
|
|
||||||
|
```shell
|
||||||
|
conda install {package_name}
|
||||||
|
conda install {package_name}={package_version}
|
||||||
|
conda install -c {channel} {package_name}
|
||||||
|
```
|
||||||
|
|
||||||
|
| Parameter | Description |
|
||||||
|
| ----------------- | ----------- |
|
||||||
|
| `package_name` | The package name. |
|
||||||
|
| `package_version` | The package version. |
|
||||||
|
| `channel` | The channel of the package. (optional) |
|
|
@ -28,6 +28,7 @@ The following package managers are currently supported:
|
||||||
| ---- | -------- | -------------- |
|
| ---- | -------- | -------------- |
|
||||||
| [Composer]({{< relref "doc/packages/composer.en-us.md" >}}) | PHP | `composer` |
|
| [Composer]({{< relref "doc/packages/composer.en-us.md" >}}) | PHP | `composer` |
|
||||||
| [Conan]({{< relref "doc/packages/conan.en-us.md" >}}) | C++ | `conan` |
|
| [Conan]({{< relref "doc/packages/conan.en-us.md" >}}) | C++ | `conan` |
|
||||||
|
| [Conda]({{< relref "doc/packages/conda.en-us.md" >}}) | - | `conda` |
|
||||||
| [Container]({{< relref "doc/packages/container.en-us.md" >}}) | - | any OCI compliant client |
|
| [Container]({{< relref "doc/packages/container.en-us.md" >}}) | - | any OCI compliant client |
|
||||||
| [Generic]({{< relref "doc/packages/generic.en-us.md" >}}) | - | any HTTP client |
|
| [Generic]({{< relref "doc/packages/generic.en-us.md" >}}) | - | any HTTP client |
|
||||||
| [Helm]({{< relref "doc/packages/helm.en-us.md" >}}) | - | any HTTP client, `cm-push` |
|
| [Helm]({{< relref "doc/packages/helm.en-us.md" >}}) | - | any HTTP client, `cm-push` |
|
||||||
|
|
2
go.mod
2
go.mod
|
@ -26,6 +26,7 @@ require (
|
||||||
github.com/dimiro1/reply v0.0.0-20200315094148-d0136a4c9e21
|
github.com/dimiro1/reply v0.0.0-20200315094148-d0136a4c9e21
|
||||||
github.com/djherbis/buffer v1.2.0
|
github.com/djherbis/buffer v1.2.0
|
||||||
github.com/djherbis/nio/v3 v3.0.1
|
github.com/djherbis/nio/v3 v3.0.1
|
||||||
|
github.com/dsnet/compress v0.0.2-0.20210315054119-f66993602bf5
|
||||||
github.com/dustin/go-humanize v1.0.0
|
github.com/dustin/go-humanize v1.0.0
|
||||||
github.com/editorconfig/editorconfig-core-go/v2 v2.5.1
|
github.com/editorconfig/editorconfig-core-go/v2 v2.5.1
|
||||||
github.com/emersion/go-imap v1.2.1
|
github.com/emersion/go-imap v1.2.1
|
||||||
|
@ -161,7 +162,6 @@ require (
|
||||||
github.com/davecgh/go-spew v1.1.1 // indirect
|
github.com/davecgh/go-spew v1.1.1 // indirect
|
||||||
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f // indirect
|
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f // indirect
|
||||||
github.com/dlclark/regexp2 v1.7.0 // indirect
|
github.com/dlclark/regexp2 v1.7.0 // indirect
|
||||||
github.com/dsnet/compress v0.0.2-0.20210315054119-f66993602bf5 // indirect
|
|
||||||
github.com/emersion/go-sasl v0.0.0-20200509203442-7bfe0ed36a21 // indirect
|
github.com/emersion/go-sasl v0.0.0-20200509203442-7bfe0ed36a21 // indirect
|
||||||
github.com/fatih/color v1.13.0 // indirect
|
github.com/fatih/color v1.13.0 // indirect
|
||||||
github.com/felixge/httpsnoop v1.0.3 // indirect
|
github.com/felixge/httpsnoop v1.0.3 // indirect
|
||||||
|
|
63
models/packages/conda/search.go
Normal file
63
models/packages/conda/search.go
Normal file
|
@ -0,0 +1,63 @@
|
||||||
|
// Copyright 2022 The Gitea Authors. All rights reserved.
|
||||||
|
// SPDX-License-Identifier: MIT
|
||||||
|
|
||||||
|
package conda
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"code.gitea.io/gitea/models/db"
|
||||||
|
"code.gitea.io/gitea/models/packages"
|
||||||
|
conda_module "code.gitea.io/gitea/modules/packages/conda"
|
||||||
|
|
||||||
|
"xorm.io/builder"
|
||||||
|
)
|
||||||
|
|
||||||
|
type FileSearchOptions struct {
|
||||||
|
OwnerID int64
|
||||||
|
Channel string
|
||||||
|
Subdir string
|
||||||
|
Filename string
|
||||||
|
}
|
||||||
|
|
||||||
|
// SearchFiles gets all files matching the search options
|
||||||
|
func SearchFiles(ctx context.Context, opts *FileSearchOptions) ([]*packages.PackageFile, error) {
|
||||||
|
var cond builder.Cond = builder.Eq{
|
||||||
|
"package.type": packages.TypeConda,
|
||||||
|
"package.owner_id": opts.OwnerID,
|
||||||
|
"package_version.is_internal": false,
|
||||||
|
}
|
||||||
|
|
||||||
|
if opts.Filename != "" {
|
||||||
|
cond = cond.And(builder.Eq{
|
||||||
|
"package_file.lower_name": strings.ToLower(opts.Filename),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
var versionPropsCond builder.Cond = builder.Eq{
|
||||||
|
"package_property.ref_type": packages.PropertyTypePackage,
|
||||||
|
"package_property.name": conda_module.PropertyChannel,
|
||||||
|
"package_property.value": opts.Channel,
|
||||||
|
}
|
||||||
|
|
||||||
|
cond = cond.And(builder.In("package.id", builder.Select("package_property.ref_id").Where(versionPropsCond).From("package_property")))
|
||||||
|
|
||||||
|
var filePropsCond builder.Cond = builder.Eq{
|
||||||
|
"package_property.ref_type": packages.PropertyTypeFile,
|
||||||
|
"package_property.name": conda_module.PropertySubdir,
|
||||||
|
"package_property.value": opts.Subdir,
|
||||||
|
}
|
||||||
|
|
||||||
|
cond = cond.And(builder.In("package_file.id", builder.Select("package_property.ref_id").Where(filePropsCond).From("package_property")))
|
||||||
|
|
||||||
|
sess := db.GetEngine(ctx).
|
||||||
|
Select("package_file.*").
|
||||||
|
Table("package_file").
|
||||||
|
Join("INNER", "package_version", "package_version.id = package_file.version_id").
|
||||||
|
Join("INNER", "package", "package.id = package_version.package_id").
|
||||||
|
Where(cond)
|
||||||
|
|
||||||
|
pfs := make([]*packages.PackageFile, 0, 10)
|
||||||
|
return pfs, sess.Find(&pfs)
|
||||||
|
}
|
|
@ -13,6 +13,7 @@ import (
|
||||||
"code.gitea.io/gitea/modules/json"
|
"code.gitea.io/gitea/modules/json"
|
||||||
"code.gitea.io/gitea/modules/packages/composer"
|
"code.gitea.io/gitea/modules/packages/composer"
|
||||||
"code.gitea.io/gitea/modules/packages/conan"
|
"code.gitea.io/gitea/modules/packages/conan"
|
||||||
|
"code.gitea.io/gitea/modules/packages/conda"
|
||||||
"code.gitea.io/gitea/modules/packages/container"
|
"code.gitea.io/gitea/modules/packages/container"
|
||||||
"code.gitea.io/gitea/modules/packages/helm"
|
"code.gitea.io/gitea/modules/packages/helm"
|
||||||
"code.gitea.io/gitea/modules/packages/maven"
|
"code.gitea.io/gitea/modules/packages/maven"
|
||||||
|
@ -132,6 +133,8 @@ func GetPackageDescriptor(ctx context.Context, pv *PackageVersion) (*PackageDesc
|
||||||
metadata = &composer.Metadata{}
|
metadata = &composer.Metadata{}
|
||||||
case TypeConan:
|
case TypeConan:
|
||||||
metadata = &conan.Metadata{}
|
metadata = &conan.Metadata{}
|
||||||
|
case TypeConda:
|
||||||
|
metadata = &conda.VersionMetadata{}
|
||||||
case TypeContainer:
|
case TypeContainer:
|
||||||
metadata = &container.Metadata{}
|
metadata = &container.Metadata{}
|
||||||
case TypeGeneric:
|
case TypeGeneric:
|
||||||
|
|
|
@ -32,6 +32,7 @@ type Type string
|
||||||
const (
|
const (
|
||||||
TypeComposer Type = "composer"
|
TypeComposer Type = "composer"
|
||||||
TypeConan Type = "conan"
|
TypeConan Type = "conan"
|
||||||
|
TypeConda Type = "conda"
|
||||||
TypeContainer Type = "container"
|
TypeContainer Type = "container"
|
||||||
TypeGeneric Type = "generic"
|
TypeGeneric Type = "generic"
|
||||||
TypeHelm Type = "helm"
|
TypeHelm Type = "helm"
|
||||||
|
@ -47,6 +48,7 @@ const (
|
||||||
var TypeList = []Type{
|
var TypeList = []Type{
|
||||||
TypeComposer,
|
TypeComposer,
|
||||||
TypeConan,
|
TypeConan,
|
||||||
|
TypeConda,
|
||||||
TypeContainer,
|
TypeContainer,
|
||||||
TypeGeneric,
|
TypeGeneric,
|
||||||
TypeHelm,
|
TypeHelm,
|
||||||
|
@ -66,6 +68,8 @@ func (pt Type) Name() string {
|
||||||
return "Composer"
|
return "Composer"
|
||||||
case TypeConan:
|
case TypeConan:
|
||||||
return "Conan"
|
return "Conan"
|
||||||
|
case TypeConda:
|
||||||
|
return "Conda"
|
||||||
case TypeContainer:
|
case TypeContainer:
|
||||||
return "Container"
|
return "Container"
|
||||||
case TypeGeneric:
|
case TypeGeneric:
|
||||||
|
@ -97,6 +101,8 @@ func (pt Type) SVGName() string {
|
||||||
return "gitea-composer"
|
return "gitea-composer"
|
||||||
case TypeConan:
|
case TypeConan:
|
||||||
return "gitea-conan"
|
return "gitea-conan"
|
||||||
|
case TypeConda:
|
||||||
|
return "gitea-conda"
|
||||||
case TypeContainer:
|
case TypeContainer:
|
||||||
return "octicon-container"
|
return "octicon-container"
|
||||||
case TypeGeneric:
|
case TypeGeneric:
|
||||||
|
|
243
modules/packages/conda/metadata.go
Normal file
243
modules/packages/conda/metadata.go
Normal file
|
@ -0,0 +1,243 @@
|
||||||
|
// Copyright 2022 The Gitea Authors. All rights reserved.
|
||||||
|
// SPDX-License-Identifier: MIT
|
||||||
|
|
||||||
|
package conda
|
||||||
|
|
||||||
|
import (
|
||||||
|
"archive/tar"
|
||||||
|
"archive/zip"
|
||||||
|
"compress/bzip2"
|
||||||
|
"io"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"code.gitea.io/gitea/modules/json"
|
||||||
|
"code.gitea.io/gitea/modules/util"
|
||||||
|
"code.gitea.io/gitea/modules/validation"
|
||||||
|
|
||||||
|
"github.com/klauspost/compress/zstd"
|
||||||
|
)
|
||||||
|
|
||||||
|
var (
|
||||||
|
ErrInvalidStructure = util.SilentWrap{Message: "package structure is invalid", Err: util.ErrInvalidArgument}
|
||||||
|
ErrInvalidName = util.SilentWrap{Message: "package name is invalid", Err: util.ErrInvalidArgument}
|
||||||
|
ErrInvalidVersion = util.SilentWrap{Message: "package version is invalid", Err: util.ErrInvalidArgument}
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
PropertyName = "conda.name"
|
||||||
|
PropertyChannel = "conda.channel"
|
||||||
|
PropertySubdir = "conda.subdir"
|
||||||
|
PropertyMetadata = "conda.metdata"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Package represents a Conda package
|
||||||
|
type Package struct {
|
||||||
|
Name string
|
||||||
|
Version string
|
||||||
|
Subdir string
|
||||||
|
VersionMetadata *VersionMetadata
|
||||||
|
FileMetadata *FileMetadata
|
||||||
|
}
|
||||||
|
|
||||||
|
// VersionMetadata represents the metadata of a Conda package
|
||||||
|
type VersionMetadata struct {
|
||||||
|
Description string `json:"description,omitempty"`
|
||||||
|
Summary string `json:"summary,omitempty"`
|
||||||
|
ProjectURL string `json:"project_url,omitempty"`
|
||||||
|
RepositoryURL string `json:"repository_url,omitempty"`
|
||||||
|
DocumentationURL string `json:"documentation_url,omitempty"`
|
||||||
|
License string `json:"license,omitempty"`
|
||||||
|
LicenseFamily string `json:"license_family,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// FileMetadata represents the metadata of a Conda package file
|
||||||
|
type FileMetadata struct {
|
||||||
|
IsCondaPackage bool `json:"is_conda"`
|
||||||
|
Architecture string `json:"architecture,omitempty"`
|
||||||
|
NoArch string `json:"noarch,omitempty"`
|
||||||
|
Build string `json:"build,omitempty"`
|
||||||
|
BuildNumber int64 `json:"build_number,omitempty"`
|
||||||
|
Dependencies []string `json:"dependencies,omitempty"`
|
||||||
|
Platform string `json:"platform,omitempty"`
|
||||||
|
Timestamp int64 `json:"timestamp,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type index struct {
|
||||||
|
Name string `json:"name"`
|
||||||
|
Version string `json:"version"`
|
||||||
|
Architecture string `json:"arch"`
|
||||||
|
NoArch string `json:"noarch"`
|
||||||
|
Build string `json:"build"`
|
||||||
|
BuildNumber int64 `json:"build_number"`
|
||||||
|
Dependencies []string `json:"depends"`
|
||||||
|
License string `json:"license"`
|
||||||
|
LicenseFamily string `json:"license_family"`
|
||||||
|
Platform string `json:"platform"`
|
||||||
|
Subdir string `json:"subdir"`
|
||||||
|
Timestamp int64 `json:"timestamp"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type about struct {
|
||||||
|
Description string `json:"description"`
|
||||||
|
Summary string `json:"summary"`
|
||||||
|
ProjectURL string `json:"home"`
|
||||||
|
RepositoryURL string `json:"dev_url"`
|
||||||
|
DocumentationURL string `json:"doc_url"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type ReaderAndReaderAt interface {
|
||||||
|
io.Reader
|
||||||
|
io.ReaderAt
|
||||||
|
}
|
||||||
|
|
||||||
|
// ParsePackageBZ2 parses the Conda package file compressed with bzip2
|
||||||
|
func ParsePackageBZ2(r io.Reader) (*Package, error) {
|
||||||
|
gzr := bzip2.NewReader(r)
|
||||||
|
|
||||||
|
return parsePackageTar(gzr)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ParsePackageConda parses the Conda package file compressed with zip and zstd
|
||||||
|
func ParsePackageConda(r io.ReaderAt, size int64) (*Package, error) {
|
||||||
|
zr, err := zip.NewReader(r, size)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, file := range zr.File {
|
||||||
|
if strings.HasPrefix(file.Name, "info-") && strings.HasSuffix(file.Name, ".tar.zst") {
|
||||||
|
f, err := zr.Open(file.Name)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
defer f.Close()
|
||||||
|
|
||||||
|
dec, err := zstd.NewReader(f)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
defer dec.Close()
|
||||||
|
|
||||||
|
p, err := parsePackageTar(dec)
|
||||||
|
if p != nil {
|
||||||
|
p.FileMetadata.IsCondaPackage = true
|
||||||
|
}
|
||||||
|
return p, err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil, ErrInvalidStructure
|
||||||
|
}
|
||||||
|
|
||||||
|
func parsePackageTar(r io.Reader) (*Package, error) {
|
||||||
|
var i *index
|
||||||
|
var a *about
|
||||||
|
|
||||||
|
tr := tar.NewReader(r)
|
||||||
|
for {
|
||||||
|
hdr, err := tr.Next()
|
||||||
|
if err == io.EOF {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
if hdr.Typeflag != tar.TypeReg {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
if hdr.Name == "info/index.json" {
|
||||||
|
if err := json.NewDecoder(tr).Decode(&i); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
if !checkName(i.Name) {
|
||||||
|
return nil, ErrInvalidName
|
||||||
|
}
|
||||||
|
|
||||||
|
if !checkVersion(i.Version) {
|
||||||
|
return nil, ErrInvalidVersion
|
||||||
|
}
|
||||||
|
|
||||||
|
if a != nil {
|
||||||
|
break // stop loop if both files were found
|
||||||
|
}
|
||||||
|
} else if hdr.Name == "info/about.json" {
|
||||||
|
if err := json.NewDecoder(tr).Decode(&a); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
if !validation.IsValidURL(a.ProjectURL) {
|
||||||
|
a.ProjectURL = ""
|
||||||
|
}
|
||||||
|
if !validation.IsValidURL(a.RepositoryURL) {
|
||||||
|
a.RepositoryURL = ""
|
||||||
|
}
|
||||||
|
if !validation.IsValidURL(a.DocumentationURL) {
|
||||||
|
a.DocumentationURL = ""
|
||||||
|
}
|
||||||
|
|
||||||
|
if i != nil {
|
||||||
|
break // stop loop if both files were found
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if i == nil {
|
||||||
|
return nil, ErrInvalidStructure
|
||||||
|
}
|
||||||
|
if a == nil {
|
||||||
|
a = &about{}
|
||||||
|
}
|
||||||
|
|
||||||
|
return &Package{
|
||||||
|
Name: i.Name,
|
||||||
|
Version: i.Version,
|
||||||
|
Subdir: i.Subdir,
|
||||||
|
VersionMetadata: &VersionMetadata{
|
||||||
|
License: i.License,
|
||||||
|
LicenseFamily: i.LicenseFamily,
|
||||||
|
Description: a.Description,
|
||||||
|
Summary: a.Summary,
|
||||||
|
ProjectURL: a.ProjectURL,
|
||||||
|
RepositoryURL: a.RepositoryURL,
|
||||||
|
DocumentationURL: a.DocumentationURL,
|
||||||
|
},
|
||||||
|
FileMetadata: &FileMetadata{
|
||||||
|
Architecture: i.Architecture,
|
||||||
|
NoArch: i.NoArch,
|
||||||
|
Build: i.Build,
|
||||||
|
BuildNumber: i.BuildNumber,
|
||||||
|
Dependencies: i.Dependencies,
|
||||||
|
Platform: i.Platform,
|
||||||
|
Timestamp: i.Timestamp,
|
||||||
|
},
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// https://github.com/conda/conda-build/blob/db9a728a9e4e6cfc895637ca3221117970fc2663/conda_build/metadata.py#L1393
|
||||||
|
func checkName(name string) bool {
|
||||||
|
if name == "" {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
if name != strings.ToLower(name) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
return !checkBadCharacters(name, "!")
|
||||||
|
}
|
||||||
|
|
||||||
|
// https://github.com/conda/conda-build/blob/db9a728a9e4e6cfc895637ca3221117970fc2663/conda_build/metadata.py#L1403
|
||||||
|
func checkVersion(version string) bool {
|
||||||
|
if version == "" {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
return !checkBadCharacters(version, "-")
|
||||||
|
}
|
||||||
|
|
||||||
|
func checkBadCharacters(s, additional string) bool {
|
||||||
|
if strings.ContainsAny(s, "=@#$%^&*:;\"'\\|<>?/ ") {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
return strings.ContainsAny(s, additional)
|
||||||
|
}
|
150
modules/packages/conda/metadata_test.go
Normal file
150
modules/packages/conda/metadata_test.go
Normal file
|
@ -0,0 +1,150 @@
|
||||||
|
// Copyright 2022 The Gitea Authors. All rights reserved.
|
||||||
|
// SPDX-License-Identifier: MIT
|
||||||
|
|
||||||
|
package conda
|
||||||
|
|
||||||
|
import (
|
||||||
|
"archive/tar"
|
||||||
|
"archive/zip"
|
||||||
|
"bytes"
|
||||||
|
"io"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/dsnet/compress/bzip2"
|
||||||
|
"github.com/klauspost/compress/zstd"
|
||||||
|
"github.com/stretchr/testify/assert"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
packageName = "gitea"
|
||||||
|
packageVersion = "1.0.1"
|
||||||
|
description = "Package Description"
|
||||||
|
projectURL = "https://gitea.io"
|
||||||
|
repositoryURL = "https://gitea.io/gitea/gitea"
|
||||||
|
documentationURL = "https://docs.gitea.io"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestParsePackage(t *testing.T) {
|
||||||
|
createArchive := func(files map[string][]byte) *bytes.Buffer {
|
||||||
|
var buf bytes.Buffer
|
||||||
|
tw := tar.NewWriter(&buf)
|
||||||
|
for filename, content := range files {
|
||||||
|
hdr := &tar.Header{
|
||||||
|
Name: filename,
|
||||||
|
Mode: 0o600,
|
||||||
|
Size: int64(len(content)),
|
||||||
|
}
|
||||||
|
tw.WriteHeader(hdr)
|
||||||
|
tw.Write(content)
|
||||||
|
}
|
||||||
|
tw.Close()
|
||||||
|
return &buf
|
||||||
|
}
|
||||||
|
|
||||||
|
t.Run("MissingIndexFile", func(t *testing.T) {
|
||||||
|
buf := createArchive(map[string][]byte{"dummy.txt": {}})
|
||||||
|
|
||||||
|
p, err := parsePackageTar(buf)
|
||||||
|
assert.Nil(t, p)
|
||||||
|
assert.ErrorIs(t, err, ErrInvalidStructure)
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("MissingAboutFile", func(t *testing.T) {
|
||||||
|
buf := createArchive(map[string][]byte{"info/index.json": []byte(`{"name":"name","version":"1.0"}`)})
|
||||||
|
|
||||||
|
p, err := parsePackageTar(buf)
|
||||||
|
assert.NotNil(t, p)
|
||||||
|
assert.NoError(t, err)
|
||||||
|
|
||||||
|
assert.Equal(t, "name", p.Name)
|
||||||
|
assert.Equal(t, "1.0", p.Version)
|
||||||
|
assert.Empty(t, p.VersionMetadata.ProjectURL)
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("InvalidName", func(t *testing.T) {
|
||||||
|
for _, name := range []string{"", "name!", "nAMe"} {
|
||||||
|
buf := createArchive(map[string][]byte{"info/index.json": []byte(`{"name":"` + name + `","version":"1.0"}`)})
|
||||||
|
|
||||||
|
p, err := parsePackageTar(buf)
|
||||||
|
assert.Nil(t, p)
|
||||||
|
assert.ErrorIs(t, err, ErrInvalidName)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("InvalidVersion", func(t *testing.T) {
|
||||||
|
for _, version := range []string{"", "1.0-2"} {
|
||||||
|
buf := createArchive(map[string][]byte{"info/index.json": []byte(`{"name":"name","version":"` + version + `"}`)})
|
||||||
|
|
||||||
|
p, err := parsePackageTar(buf)
|
||||||
|
assert.Nil(t, p)
|
||||||
|
assert.ErrorIs(t, err, ErrInvalidVersion)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("Valid", func(t *testing.T) {
|
||||||
|
buf := createArchive(map[string][]byte{
|
||||||
|
"info/index.json": []byte(`{"name":"` + packageName + `","version":"` + packageVersion + `","subdir":"linux-64"}`),
|
||||||
|
"info/about.json": []byte(`{"description":"` + description + `","dev_url":"` + repositoryURL + `","doc_url":"` + documentationURL + `","home":"` + projectURL + `"}`),
|
||||||
|
})
|
||||||
|
|
||||||
|
p, err := parsePackageTar(buf)
|
||||||
|
assert.NotNil(t, p)
|
||||||
|
assert.NoError(t, err)
|
||||||
|
|
||||||
|
assert.Equal(t, packageName, p.Name)
|
||||||
|
assert.Equal(t, packageVersion, p.Version)
|
||||||
|
assert.Equal(t, "linux-64", p.Subdir)
|
||||||
|
assert.Equal(t, description, p.VersionMetadata.Description)
|
||||||
|
assert.Equal(t, projectURL, p.VersionMetadata.ProjectURL)
|
||||||
|
assert.Equal(t, repositoryURL, p.VersionMetadata.RepositoryURL)
|
||||||
|
assert.Equal(t, documentationURL, p.VersionMetadata.DocumentationURL)
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run(".tar.bz2", func(t *testing.T) {
|
||||||
|
tarArchive := createArchive(map[string][]byte{
|
||||||
|
"info/index.json": []byte(`{"name":"` + packageName + `","version":"` + packageVersion + `"}`),
|
||||||
|
})
|
||||||
|
|
||||||
|
var buf bytes.Buffer
|
||||||
|
bw, _ := bzip2.NewWriter(&buf, nil)
|
||||||
|
io.Copy(bw, tarArchive)
|
||||||
|
bw.Close()
|
||||||
|
|
||||||
|
br := bytes.NewReader(buf.Bytes())
|
||||||
|
|
||||||
|
p, err := ParsePackageBZ2(br)
|
||||||
|
assert.NotNil(t, p)
|
||||||
|
assert.NoError(t, err)
|
||||||
|
|
||||||
|
assert.Equal(t, packageName, p.Name)
|
||||||
|
assert.Equal(t, packageVersion, p.Version)
|
||||||
|
assert.False(t, p.FileMetadata.IsCondaPackage)
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run(".conda", func(t *testing.T) {
|
||||||
|
tarArchive := createArchive(map[string][]byte{
|
||||||
|
"info/index.json": []byte(`{"name":"` + packageName + `","version":"` + packageVersion + `"}`),
|
||||||
|
})
|
||||||
|
|
||||||
|
var infoBuf bytes.Buffer
|
||||||
|
zsw, _ := zstd.NewWriter(&infoBuf)
|
||||||
|
io.Copy(zsw, tarArchive)
|
||||||
|
zsw.Close()
|
||||||
|
|
||||||
|
var buf bytes.Buffer
|
||||||
|
zpw := zip.NewWriter(&buf)
|
||||||
|
w, _ := zpw.Create("info-x.tar.zst")
|
||||||
|
w.Write(infoBuf.Bytes())
|
||||||
|
zpw.Close()
|
||||||
|
|
||||||
|
br := bytes.NewReader(buf.Bytes())
|
||||||
|
|
||||||
|
p, err := ParsePackageConda(br, int64(br.Len()))
|
||||||
|
assert.NotNil(t, p)
|
||||||
|
assert.NoError(t, err)
|
||||||
|
|
||||||
|
assert.Equal(t, packageName, p.Name)
|
||||||
|
assert.Equal(t, packageVersion, p.Version)
|
||||||
|
assert.True(t, p.FileMetadata.IsCondaPackage)
|
||||||
|
})
|
||||||
|
}
|
|
@ -27,6 +27,7 @@ var (
|
||||||
LimitTotalOwnerSize int64
|
LimitTotalOwnerSize int64
|
||||||
LimitSizeComposer int64
|
LimitSizeComposer int64
|
||||||
LimitSizeConan int64
|
LimitSizeConan int64
|
||||||
|
LimitSizeConda int64
|
||||||
LimitSizeContainer int64
|
LimitSizeContainer int64
|
||||||
LimitSizeGeneric int64
|
LimitSizeGeneric int64
|
||||||
LimitSizeHelm int64
|
LimitSizeHelm int64
|
||||||
|
@ -66,6 +67,7 @@ func newPackages() {
|
||||||
Packages.LimitTotalOwnerSize = mustBytes(sec, "LIMIT_TOTAL_OWNER_SIZE")
|
Packages.LimitTotalOwnerSize = mustBytes(sec, "LIMIT_TOTAL_OWNER_SIZE")
|
||||||
Packages.LimitSizeComposer = mustBytes(sec, "LIMIT_SIZE_COMPOSER")
|
Packages.LimitSizeComposer = mustBytes(sec, "LIMIT_SIZE_COMPOSER")
|
||||||
Packages.LimitSizeConan = mustBytes(sec, "LIMIT_SIZE_CONAN")
|
Packages.LimitSizeConan = mustBytes(sec, "LIMIT_SIZE_CONAN")
|
||||||
|
Packages.LimitSizeConda = mustBytes(sec, "LIMIT_SIZE_CONDA")
|
||||||
Packages.LimitSizeContainer = mustBytes(sec, "LIMIT_SIZE_CONTAINER")
|
Packages.LimitSizeContainer = mustBytes(sec, "LIMIT_SIZE_CONTAINER")
|
||||||
Packages.LimitSizeGeneric = mustBytes(sec, "LIMIT_SIZE_GENERIC")
|
Packages.LimitSizeGeneric = mustBytes(sec, "LIMIT_SIZE_GENERIC")
|
||||||
Packages.LimitSizeHelm = mustBytes(sec, "LIMIT_SIZE_HELM")
|
Packages.LimitSizeHelm = mustBytes(sec, "LIMIT_SIZE_HELM")
|
||||||
|
|
|
@ -3159,6 +3159,11 @@ conan.details.repository = Repository
|
||||||
conan.registry = Setup this registry from the command line:
|
conan.registry = Setup this registry from the command line:
|
||||||
conan.install = To install the package using Conan, run the following command:
|
conan.install = To install the package using Conan, run the following command:
|
||||||
conan.documentation = For more information on the Conan registry, see <a target="_blank" rel="noopener noreferrer" href="https://docs.gitea.io/en-us/packages/conan/">the documentation</a>.
|
conan.documentation = For more information on the Conan registry, see <a target="_blank" rel="noopener noreferrer" href="https://docs.gitea.io/en-us/packages/conan/">the documentation</a>.
|
||||||
|
conda.registry = Setup this registry as a Conda repository in your <code>.condarc</code> file:
|
||||||
|
conda.install = To install the package using Conda, run the following command:
|
||||||
|
conda.documentation = For more information on the Conda registry, see <a target="_blank" rel="noopener noreferrer" href="https://docs.gitea.io/en-us/packages/conda/">the documentation</a>.
|
||||||
|
conda.details.repository_site = Repository Site
|
||||||
|
conda.details.documentation_site = Documentation Site
|
||||||
container.details.type = Image Type
|
container.details.type = Image Type
|
||||||
container.details.platform = Platform
|
container.details.platform = Platform
|
||||||
container.details.repository_site = Repository Site
|
container.details.repository_site = Repository Site
|
||||||
|
|
1
public/img/svg/gitea-conda.svg
Normal file
1
public/img/svg/gitea-conda.svg
Normal file
|
@ -0,0 +1 @@
|
||||||
|
<svg viewBox="0 0 32 32" class="svg gitea-conda" width="16" height="16" aria-hidden="true"><path fill="#43b02a" fill-rule="evenodd" stroke="#43b02a" stroke-width=".068" d="M16.559 8.137a7.2 7.2 0 0 0-1.234-1.708 7.586 7.586 0 0 0-.19 2.183 5.161 5.161 0 0 1 1.424-.475ZM13.617 9.466a7.992 7.992 0 0 0-1.993-1.2 8.123 8.123 0 0 0 .885 2.183c0 .063.443-.475 1.108-.981ZM17.445 7.188a9.143 9.143 0 0 1 1.3-2.246A7.585 7.585 0 0 0 17 2.854a8.35 8.35 0 0 0-1.3 2.278 8.451 8.451 0 0 1 1.74 2.056ZM11.592 11.744a10.276 10.276 0 0 0-2.692-.158 7.478 7.478 0 0 0 1.93 1.9 6.858 6.858 0 0 1 .759-1.74zM6.878 15.161a7.44 7.44 0 0 1 2.942-1.139 10.019 10.019 0 0 1-2.056-2.278 7.639 7.639 0 0 0-2.847 1.2 7.11 7.11 0 0 0 1.961 2.215zM10.516 14.876a6.16 6.16 0 0 0-2.815.886 9.936 9.936 0 0 0 2.815 1.2 7.683 7.683 0 0 1 0-2.088zM14.281 5.543A7.839 7.839 0 0 0 11.592 4.4 8.361 8.361 0 0 0 11.4 7a8.875 8.875 0 0 1 2.47 1.264 10.292 10.292 0 0 1 .411-2.721ZM24.025 3.234a20.488 20.488 0 0 1 .917 4.112 6.823 6.823 0 0 0-3.068 1.519 7.443 7.443 0 0 1 1.55 1.044 1.351 1.351 0 0 0 1.645.316 36.938 36.938 0 0 0 2.721-2.72 1.273 1.273 0 0 0-.159-1.835 20.521 20.521 0 0 0-3.606-2.436ZM4.379 12.06a8.67 8.67 0 0 1 2.847-1.26 7.763 7.763 0 0 1-.759-2.974 14.687 14.687 0 0 0-2.088 4.234ZM11.339 10.668a9.991 9.991 0 0 1-.949-2.784 7.928 7.928 0 0 0-2.911-.126 7.312 7.312 0 0 0 .791 2.879 9.664 9.664 0 0 1 3.069.031ZM6.119 15.73a8.894 8.894 0 0 1-2.025-2.373 14.208 14.208 0 0 0-.063 4.9 8.522 8.522 0 0 1 2.088-2.527Z"/><path fill="#43b02a" fill-rule="evenodd" stroke="#43b02a" stroke-width=".068" d="M22.538 3.487A7.581 7.581 0 0 0 20.323 5.1a11.789 11.789 0 0 1 .823 2.5 9.775 9.775 0 0 1 2.309-1.329 6.593 6.593 0 0 0-.917-2.784ZM19.374 6.3a8.608 8.608 0 0 0-.822 1.676 9.645 9.645 0 0 1 1.329.19 7.568 7.568 0 0 0-.507-1.866ZM19.659 3.9a9.577 9.577 0 0 1 2.056-1.487A15.38 15.38 0 0 0 18.046 2a9.709 9.709 0 0 1 1.613 1.9Z"/><path fill="#43b02a" d="M27.378 23.892c-1.993-1.9-2.4-3.132-4.081-1.835a7.837 7.837 0 0 1-12.591-4.144A10.179 10.179 0 0 1 6.878 16.3a9.427 9.427 0 0 0-2.562 3.321h-.032C7.163 30.5 21.178 33.035 27.663 26.233c1.076-1.139.095-1.933-.285-2.341ZM6.309 20.855a7.559 7.559 0 0 1 .917-2.025 6.872 6.872 0 0 0 2.151.538c1.013 2.689 4.556 6.264 8.922 6.264a9.632 9.632 0 0 0 6.3-2.309 12.841 12.841 0 0 1 1.772 1.771c.095.127.095.159.095.159-5.766 5.03-15.538 4.302-20.157-4.398Z"/><path fill="#43b02a" fill-rule="evenodd" stroke="#43b02a" stroke-width=".067" d="M10.67 4.11a19.934 19.934 0 0 0-.214 2.509 10.512 10.512 0 0 0-2.689-.093A18 18 0 0 1 10.67 4.11ZM12.26 3.274a9.107 9.107 0 0 1 2.445 1.053 14.083 14.083 0 0 1 1.253-2.137 12.106 12.106 0 0 0-3.698 1.084z"/></svg>
|
After Width: | Height: | Size: 2.6 KiB |
|
@ -16,6 +16,7 @@ import (
|
||||||
"code.gitea.io/gitea/modules/web"
|
"code.gitea.io/gitea/modules/web"
|
||||||
"code.gitea.io/gitea/routers/api/packages/composer"
|
"code.gitea.io/gitea/routers/api/packages/composer"
|
||||||
"code.gitea.io/gitea/routers/api/packages/conan"
|
"code.gitea.io/gitea/routers/api/packages/conan"
|
||||||
|
"code.gitea.io/gitea/routers/api/packages/conda"
|
||||||
"code.gitea.io/gitea/routers/api/packages/container"
|
"code.gitea.io/gitea/routers/api/packages/container"
|
||||||
"code.gitea.io/gitea/routers/api/packages/generic"
|
"code.gitea.io/gitea/routers/api/packages/generic"
|
||||||
"code.gitea.io/gitea/routers/api/packages/helm"
|
"code.gitea.io/gitea/routers/api/packages/helm"
|
||||||
|
@ -167,6 +168,43 @@ func CommonRoutes(ctx gocontext.Context) *web.Route {
|
||||||
})
|
})
|
||||||
})
|
})
|
||||||
}, reqPackageAccess(perm.AccessModeRead))
|
}, reqPackageAccess(perm.AccessModeRead))
|
||||||
|
r.Group("/conda", func() {
|
||||||
|
var (
|
||||||
|
downloadPattern = regexp.MustCompile(`\A(.+/)?(.+)/((?:[^/]+(?:\.tar\.bz2|\.conda))|(?:current_)?repodata\.json(?:\.bz2)?)\z`)
|
||||||
|
uploadPattern = regexp.MustCompile(`\A(.+/)?([^/]+(?:\.tar\.bz2|\.conda))\z`)
|
||||||
|
)
|
||||||
|
|
||||||
|
r.Get("/*", func(ctx *context.Context) {
|
||||||
|
m := downloadPattern.FindStringSubmatch(ctx.Params("*"))
|
||||||
|
if len(m) == 0 {
|
||||||
|
ctx.Status(http.StatusNotFound)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
ctx.SetParams("channel", strings.TrimSuffix(m[1], "/"))
|
||||||
|
ctx.SetParams("architecture", m[2])
|
||||||
|
ctx.SetParams("filename", m[3])
|
||||||
|
|
||||||
|
switch m[3] {
|
||||||
|
case "repodata.json", "repodata.json.bz2", "current_repodata.json", "current_repodata.json.bz2":
|
||||||
|
conda.EnumeratePackages(ctx)
|
||||||
|
default:
|
||||||
|
conda.DownloadPackageFile(ctx)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
r.Put("/*", reqPackageAccess(perm.AccessModeWrite), func(ctx *context.Context) {
|
||||||
|
m := uploadPattern.FindStringSubmatch(ctx.Params("*"))
|
||||||
|
if len(m) == 0 {
|
||||||
|
ctx.Status(http.StatusNotFound)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
ctx.SetParams("channel", strings.TrimSuffix(m[1], "/"))
|
||||||
|
ctx.SetParams("filename", m[2])
|
||||||
|
|
||||||
|
conda.UploadPackageFile(ctx)
|
||||||
|
})
|
||||||
|
}, reqPackageAccess(perm.AccessModeRead))
|
||||||
r.Group("/generic", func() {
|
r.Group("/generic", func() {
|
||||||
r.Group("/{packagename}/{packageversion}", func() {
|
r.Group("/{packagename}/{packageversion}", func() {
|
||||||
r.Delete("", reqPackageAccess(perm.AccessModeWrite), generic.DeletePackage)
|
r.Delete("", reqPackageAccess(perm.AccessModeWrite), generic.DeletePackage)
|
||||||
|
|
306
routers/api/packages/conda/conda.go
Normal file
306
routers/api/packages/conda/conda.go
Normal file
|
@ -0,0 +1,306 @@
|
||||||
|
// Copyright 2022 The Gitea Authors. All rights reserved.
|
||||||
|
// SPDX-License-Identifier: MIT
|
||||||
|
|
||||||
|
package conda
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"net/http"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
packages_model "code.gitea.io/gitea/models/packages"
|
||||||
|
conda_model "code.gitea.io/gitea/models/packages/conda"
|
||||||
|
"code.gitea.io/gitea/modules/context"
|
||||||
|
"code.gitea.io/gitea/modules/json"
|
||||||
|
"code.gitea.io/gitea/modules/log"
|
||||||
|
packages_module "code.gitea.io/gitea/modules/packages"
|
||||||
|
conda_module "code.gitea.io/gitea/modules/packages/conda"
|
||||||
|
"code.gitea.io/gitea/modules/util"
|
||||||
|
"code.gitea.io/gitea/routers/api/packages/helper"
|
||||||
|
packages_service "code.gitea.io/gitea/services/packages"
|
||||||
|
|
||||||
|
"github.com/dsnet/compress/bzip2"
|
||||||
|
)
|
||||||
|
|
||||||
|
func apiError(ctx *context.Context, status int, obj interface{}) {
|
||||||
|
helper.LogAndProcessError(ctx, status, obj, func(message string) {
|
||||||
|
ctx.JSON(status, struct {
|
||||||
|
Reason string `json:"reason"`
|
||||||
|
Message string `json:"message"`
|
||||||
|
}{
|
||||||
|
Reason: http.StatusText(status),
|
||||||
|
Message: message,
|
||||||
|
})
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func EnumeratePackages(ctx *context.Context) {
|
||||||
|
type Info struct {
|
||||||
|
Subdir string `json:"subdir"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type PackageInfo struct {
|
||||||
|
Name string `json:"name"`
|
||||||
|
Version string `json:"version"`
|
||||||
|
NoArch string `json:"noarch"`
|
||||||
|
Subdir string `json:"subdir"`
|
||||||
|
Timestamp int64 `json:"timestamp"`
|
||||||
|
Build string `json:"build"`
|
||||||
|
BuildNumber int64 `json:"build_number"`
|
||||||
|
Dependencies []string `json:"depends"`
|
||||||
|
License string `json:"license"`
|
||||||
|
LicenseFamily string `json:"license_family"`
|
||||||
|
HashMD5 string `json:"md5"`
|
||||||
|
HashSHA256 string `json:"sha256"`
|
||||||
|
Size int64 `json:"size"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type RepoData struct {
|
||||||
|
Info Info `json:"info"`
|
||||||
|
Packages map[string]*PackageInfo `json:"packages"`
|
||||||
|
PackagesConda map[string]*PackageInfo `json:"packages.conda"`
|
||||||
|
Removed map[string]*PackageInfo `json:"removed"`
|
||||||
|
}
|
||||||
|
|
||||||
|
repoData := &RepoData{
|
||||||
|
Info: Info{
|
||||||
|
Subdir: ctx.Params("architecture"),
|
||||||
|
},
|
||||||
|
Packages: make(map[string]*PackageInfo),
|
||||||
|
PackagesConda: make(map[string]*PackageInfo),
|
||||||
|
Removed: make(map[string]*PackageInfo),
|
||||||
|
}
|
||||||
|
|
||||||
|
pfs, err := conda_model.SearchFiles(ctx, &conda_model.FileSearchOptions{
|
||||||
|
OwnerID: ctx.Package.Owner.ID,
|
||||||
|
Channel: ctx.Params("channel"),
|
||||||
|
Subdir: repoData.Info.Subdir,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
apiError(ctx, http.StatusInternalServerError, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(pfs) == 0 {
|
||||||
|
apiError(ctx, http.StatusNotFound, nil)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
pds := make(map[int64]*packages_model.PackageDescriptor)
|
||||||
|
|
||||||
|
for _, pf := range pfs {
|
||||||
|
pd, exists := pds[pf.VersionID]
|
||||||
|
if !exists {
|
||||||
|
pv, err := packages_model.GetVersionByID(ctx, pf.VersionID)
|
||||||
|
if err != nil {
|
||||||
|
apiError(ctx, http.StatusInternalServerError, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
pd, err = packages_model.GetPackageDescriptor(ctx, pv)
|
||||||
|
if err != nil {
|
||||||
|
apiError(ctx, http.StatusInternalServerError, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
pds[pf.VersionID] = pd
|
||||||
|
}
|
||||||
|
|
||||||
|
var pfd *packages_model.PackageFileDescriptor
|
||||||
|
for _, d := range pd.Files {
|
||||||
|
if d.File.ID == pf.ID {
|
||||||
|
pfd = d
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
var fileMetadata *conda_module.FileMetadata
|
||||||
|
if err := json.Unmarshal([]byte(pfd.Properties.GetByName(conda_module.PropertyMetadata)), &fileMetadata); err != nil {
|
||||||
|
apiError(ctx, http.StatusInternalServerError, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
versionMetadata := pd.Metadata.(*conda_module.VersionMetadata)
|
||||||
|
|
||||||
|
pi := &PackageInfo{
|
||||||
|
Name: pd.PackageProperties.GetByName(conda_module.PropertyName),
|
||||||
|
Version: pd.Version.Version,
|
||||||
|
NoArch: fileMetadata.NoArch,
|
||||||
|
Subdir: repoData.Info.Subdir,
|
||||||
|
Timestamp: fileMetadata.Timestamp,
|
||||||
|
Build: fileMetadata.Build,
|
||||||
|
BuildNumber: fileMetadata.BuildNumber,
|
||||||
|
Dependencies: fileMetadata.Dependencies,
|
||||||
|
License: versionMetadata.License,
|
||||||
|
LicenseFamily: versionMetadata.LicenseFamily,
|
||||||
|
HashMD5: pfd.Blob.HashMD5,
|
||||||
|
HashSHA256: pfd.Blob.HashSHA256,
|
||||||
|
Size: pfd.Blob.Size,
|
||||||
|
}
|
||||||
|
|
||||||
|
if fileMetadata.IsCondaPackage {
|
||||||
|
repoData.PackagesConda[pfd.File.Name] = pi
|
||||||
|
} else {
|
||||||
|
repoData.Packages[pfd.File.Name] = pi
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
resp := ctx.Resp
|
||||||
|
|
||||||
|
var w io.Writer = resp
|
||||||
|
|
||||||
|
if strings.HasSuffix(ctx.Params("filename"), ".json") {
|
||||||
|
resp.Header().Set("Content-Type", "application/json")
|
||||||
|
} else {
|
||||||
|
resp.Header().Set("Content-Type", "application/x-bzip2")
|
||||||
|
|
||||||
|
zw, err := bzip2.NewWriter(w, nil)
|
||||||
|
if err != nil {
|
||||||
|
apiError(ctx, http.StatusInternalServerError, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
defer zw.Close()
|
||||||
|
|
||||||
|
w = zw
|
||||||
|
}
|
||||||
|
|
||||||
|
resp.WriteHeader(http.StatusOK)
|
||||||
|
|
||||||
|
if err := json.NewEncoder(w).Encode(repoData); err != nil {
|
||||||
|
log.Error("JSON encode: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func UploadPackageFile(ctx *context.Context) {
|
||||||
|
upload, close, err := ctx.UploadStream()
|
||||||
|
if err != nil {
|
||||||
|
apiError(ctx, http.StatusInternalServerError, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if close {
|
||||||
|
defer upload.Close()
|
||||||
|
}
|
||||||
|
|
||||||
|
buf, err := packages_module.CreateHashedBufferFromReader(upload, 32*1024*1024)
|
||||||
|
if err != nil {
|
||||||
|
apiError(ctx, http.StatusInternalServerError, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
defer buf.Close()
|
||||||
|
|
||||||
|
var pck *conda_module.Package
|
||||||
|
if strings.HasSuffix(strings.ToLower(ctx.Params("filename")), ".tar.bz2") {
|
||||||
|
pck, err = conda_module.ParsePackageBZ2(buf)
|
||||||
|
} else {
|
||||||
|
pck, err = conda_module.ParsePackageConda(buf, buf.Size())
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
if errors.Is(err, util.ErrInvalidArgument) {
|
||||||
|
apiError(ctx, http.StatusBadRequest, err)
|
||||||
|
} else {
|
||||||
|
apiError(ctx, http.StatusInternalServerError, err)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if _, err := buf.Seek(0, io.SeekStart); err != nil {
|
||||||
|
apiError(ctx, http.StatusInternalServerError, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
fullName := pck.Name
|
||||||
|
|
||||||
|
channel := ctx.Params("channel")
|
||||||
|
if channel != "" {
|
||||||
|
fullName = channel + "/" + pck.Name
|
||||||
|
}
|
||||||
|
|
||||||
|
extension := ".tar.bz2"
|
||||||
|
if pck.FileMetadata.IsCondaPackage {
|
||||||
|
extension = ".conda"
|
||||||
|
}
|
||||||
|
|
||||||
|
fileMetadataRaw, err := json.Marshal(pck.FileMetadata)
|
||||||
|
if err != nil {
|
||||||
|
apiError(ctx, http.StatusInternalServerError, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
_, _, err = packages_service.CreatePackageOrAddFileToExisting(
|
||||||
|
&packages_service.PackageCreationInfo{
|
||||||
|
PackageInfo: packages_service.PackageInfo{
|
||||||
|
Owner: ctx.Package.Owner,
|
||||||
|
PackageType: packages_model.TypeConda,
|
||||||
|
Name: fullName,
|
||||||
|
Version: pck.Version,
|
||||||
|
},
|
||||||
|
SemverCompatible: false,
|
||||||
|
Creator: ctx.Doer,
|
||||||
|
Metadata: pck.VersionMetadata,
|
||||||
|
PackageProperties: map[string]string{
|
||||||
|
conda_module.PropertyName: pck.Name,
|
||||||
|
conda_module.PropertyChannel: channel,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
&packages_service.PackageFileCreationInfo{
|
||||||
|
PackageFileInfo: packages_service.PackageFileInfo{
|
||||||
|
Filename: fmt.Sprintf("%s-%s-%s%s", pck.Name, pck.Version, pck.FileMetadata.Build, extension),
|
||||||
|
CompositeKey: pck.Subdir,
|
||||||
|
},
|
||||||
|
Creator: ctx.Doer,
|
||||||
|
Data: buf,
|
||||||
|
IsLead: true,
|
||||||
|
Properties: map[string]string{
|
||||||
|
conda_module.PropertySubdir: pck.Subdir,
|
||||||
|
conda_module.PropertyMetadata: string(fileMetadataRaw),
|
||||||
|
},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
switch err {
|
||||||
|
case packages_model.ErrDuplicatePackageFile:
|
||||||
|
apiError(ctx, http.StatusConflict, err)
|
||||||
|
case packages_service.ErrQuotaTotalCount, packages_service.ErrQuotaTypeSize, packages_service.ErrQuotaTotalSize:
|
||||||
|
apiError(ctx, http.StatusForbidden, err)
|
||||||
|
default:
|
||||||
|
apiError(ctx, http.StatusInternalServerError, err)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
ctx.Status(http.StatusCreated)
|
||||||
|
}
|
||||||
|
|
||||||
|
func DownloadPackageFile(ctx *context.Context) {
|
||||||
|
pfs, err := conda_model.SearchFiles(ctx, &conda_model.FileSearchOptions{
|
||||||
|
OwnerID: ctx.Package.Owner.ID,
|
||||||
|
Channel: ctx.Params("channel"),
|
||||||
|
Subdir: ctx.Params("architecture"),
|
||||||
|
Filename: ctx.Params("filename"),
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
apiError(ctx, http.StatusInternalServerError, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(pfs) != 1 {
|
||||||
|
apiError(ctx, http.StatusNotFound, nil)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
pf := pfs[0]
|
||||||
|
|
||||||
|
s, _, err := packages_service.GetPackageFileStream(ctx, pf)
|
||||||
|
if err != nil {
|
||||||
|
apiError(ctx, http.StatusInternalServerError, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
defer s.Close()
|
||||||
|
|
||||||
|
ctx.ServeContent(s, &context.ServeHeaderOptions{
|
||||||
|
Filename: pf.Name,
|
||||||
|
LastModified: pf.CreatedUnix.AsLocalTime(),
|
||||||
|
})
|
||||||
|
}
|
|
@ -40,7 +40,7 @@ func ListPackages(ctx *context.APIContext) {
|
||||||
// in: query
|
// in: query
|
||||||
// description: package type filter
|
// description: package type filter
|
||||||
// type: string
|
// type: string
|
||||||
// enum: [composer, conan, container, generic, helm, maven, npm, nuget, pub, pypi, rubygems, vagrant]
|
// enum: [composer, conan, conda, container, generic, helm, maven, npm, nuget, pub, pypi, rubygems, vagrant]
|
||||||
// - name: q
|
// - name: q
|
||||||
// in: query
|
// in: query
|
||||||
// description: name filter
|
// description: name filter
|
||||||
|
|
|
@ -15,7 +15,7 @@ import (
|
||||||
type PackageCleanupRuleForm struct {
|
type PackageCleanupRuleForm struct {
|
||||||
ID int64
|
ID int64
|
||||||
Enabled bool
|
Enabled bool
|
||||||
Type string `binding:"Required;In(composer,conan,container,generic,helm,maven,npm,nuget,pub,pypi,rubygems,vagrant)"`
|
Type string `binding:"Required;In(composer,conan,conda,container,generic,helm,maven,npm,nuget,pub,pypi,rubygems,vagrant)"`
|
||||||
KeepCount int `binding:"In(0,1,5,10,25,50,100)"`
|
KeepCount int `binding:"In(0,1,5,10,25,50,100)"`
|
||||||
KeepPattern string `binding:"RegexPattern"`
|
KeepPattern string `binding:"RegexPattern"`
|
||||||
RemoveDays int `binding:"In(0,7,14,30,60,90,180)"`
|
RemoveDays int `binding:"In(0,7,14,30,60,90,180)"`
|
||||||
|
|
|
@ -339,6 +339,8 @@ func CheckSizeQuotaExceeded(ctx context.Context, doer, owner *user_model.User, p
|
||||||
typeSpecificSize = setting.Packages.LimitSizeComposer
|
typeSpecificSize = setting.Packages.LimitSizeComposer
|
||||||
case packages_model.TypeConan:
|
case packages_model.TypeConan:
|
||||||
typeSpecificSize = setting.Packages.LimitSizeConan
|
typeSpecificSize = setting.Packages.LimitSizeConan
|
||||||
|
case packages_model.TypeConda:
|
||||||
|
typeSpecificSize = setting.Packages.LimitSizeConda
|
||||||
case packages_model.TypeContainer:
|
case packages_model.TypeContainer:
|
||||||
typeSpecificSize = setting.Packages.LimitSizeContainer
|
typeSpecificSize = setting.Packages.LimitSizeContainer
|
||||||
case packages_model.TypeGeneric:
|
case packages_model.TypeGeneric:
|
||||||
|
|
30
templates/package/content/conda.tmpl
Normal file
30
templates/package/content/conda.tmpl
Normal file
|
@ -0,0 +1,30 @@
|
||||||
|
{{if eq .PackageDescriptor.Package.Type "conda"}}
|
||||||
|
<h4 class="ui top attached header">{{.locale.Tr "packages.installation"}}</h4>
|
||||||
|
<div class="ui attached segment">
|
||||||
|
<div class="ui form">
|
||||||
|
<div class="field">
|
||||||
|
<label>{{svg "octicon-code"}} {{.locale.Tr "packages.conda.registry" | Safe}}</label>
|
||||||
|
<div class="markup"><pre class="code-block"><code>channel_alias: {{AppUrl}}api/packages/{{.PackageDescriptor.Owner.Name}}/conda
|
||||||
|
channels:
|
||||||
|
  - {{AppUrl}}api/packages/{{.PackageDescriptor.Owner.Name}}/conda
|
||||||
|
default_channels:
|
||||||
|
  - {{AppUrl}}api/packages/{{.PackageDescriptor.Owner.Name}}/conda</code></pre></div>
|
||||||
|
</div>
|
||||||
|
<div class="field">
|
||||||
|
<label>{{svg "octicon-terminal"}} {{.locale.Tr "packages.conda.install"}}</label>
|
||||||
|
{{$channel := .PackageDescriptor.PackageProperties.GetByName "conda.channel"}}
|
||||||
|
<div class="markup"><pre class="code-block"><code>conda install{{if $channel}} -c {{$channel}}{{end}} {{.PackageDescriptor.PackageProperties.GetByName "conda.name"}}={{.PackageDescriptor.Version.Version}}</code></pre></div>
|
||||||
|
</div>
|
||||||
|
<div class="field">
|
||||||
|
<label>{{.locale.Tr "packages.conda.documentation" | Safe}}</label>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
{{if or .PackageDescriptor.Metadata.Description .PackageDescriptor.Metadata.Summary}}
|
||||||
|
<h4 class="ui top attached header">{{.locale.Tr "packages.about"}}</h4>
|
||||||
|
<div class="ui attached segment">
|
||||||
|
{{if .PackageDescriptor.Metadata.Description}}{{.PackageDescriptor.Metadata.Description}}{{else}}{{.PackageDescriptor.Metadata.Summary}}{{end}}
|
||||||
|
</div>
|
||||||
|
{{end}}
|
||||||
|
{{end}}
|
6
templates/package/metadata/conda.tmpl
Normal file
6
templates/package/metadata/conda.tmpl
Normal file
|
@ -0,0 +1,6 @@
|
||||||
|
{{if eq .PackageDescriptor.Package.Type "conda"}}
|
||||||
|
{{if .PackageDescriptor.Metadata.License}}<div class="item">{{svg "octicon-law" 16 "mr-3"}} {{.PackageDescriptor.Metadata.License}}</div>{{end}}
|
||||||
|
{{if .PackageDescriptor.Metadata.ProjectURL}}<div class="item">{{svg "octicon-link-external" 16 "mr-3"}} <a href="{{.PackageDescriptor.Metadata.ProjectURL}}" target="_blank" rel="noopener noreferrer me">{{.locale.Tr "packages.details.project_site"}}</a></div>{{end}}
|
||||||
|
{{if .PackageDescriptor.Metadata.RepositoryURL}}<div class="item">{{svg "octicon-link-external" 16 "mr-3"}} <a href="{{.PackageDescriptor.Metadata.RepositoryURL}}" target="_blank" rel="noopener noreferrer me">{{.locale.Tr "packages.conda.details.repository_site"}}</a></div>{{end}}
|
||||||
|
{{if .PackageDescriptor.Metadata.DocumentationURL}}<div class="item">{{svg "octicon-link-external" 16 "mr-3"}} <a href="{{.PackageDescriptor.Metadata.DocumentationURL}}" target="_blank" rel="noopener noreferrer me">{{.locale.Tr "packages.conda.details.documentation_site"}}</a></div>{{end}}
|
||||||
|
{{end}}
|
|
@ -21,6 +21,7 @@
|
||||||
<div class="twelve wide column">
|
<div class="twelve wide column">
|
||||||
{{template "package/content/composer" .}}
|
{{template "package/content/composer" .}}
|
||||||
{{template "package/content/conan" .}}
|
{{template "package/content/conan" .}}
|
||||||
|
{{template "package/content/conda" .}}
|
||||||
{{template "package/content/container" .}}
|
{{template "package/content/container" .}}
|
||||||
{{template "package/content/generic" .}}
|
{{template "package/content/generic" .}}
|
||||||
{{template "package/content/helm" .}}
|
{{template "package/content/helm" .}}
|
||||||
|
@ -44,6 +45,7 @@
|
||||||
<div class="item">{{svg "octicon-download" 16 "mr-3"}} {{.PackageDescriptor.Version.DownloadCount}}</div>
|
<div class="item">{{svg "octicon-download" 16 "mr-3"}} {{.PackageDescriptor.Version.DownloadCount}}</div>
|
||||||
{{template "package/metadata/composer" .}}
|
{{template "package/metadata/composer" .}}
|
||||||
{{template "package/metadata/conan" .}}
|
{{template "package/metadata/conan" .}}
|
||||||
|
{{template "package/metadata/conda" .}}
|
||||||
{{template "package/metadata/container" .}}
|
{{template "package/metadata/container" .}}
|
||||||
{{template "package/metadata/generic" .}}
|
{{template "package/metadata/generic" .}}
|
||||||
{{template "package/metadata/helm" .}}
|
{{template "package/metadata/helm" .}}
|
||||||
|
|
|
@ -2102,6 +2102,7 @@
|
||||||
"enum": [
|
"enum": [
|
||||||
"composer",
|
"composer",
|
||||||
"conan",
|
"conan",
|
||||||
|
"conda",
|
||||||
"container",
|
"container",
|
||||||
"generic",
|
"generic",
|
||||||
"helm",
|
"helm",
|
||||||
|
|
274
tests/integration/api_packages_conda_test.go
Normal file
274
tests/integration/api_packages_conda_test.go
Normal file
|
@ -0,0 +1,274 @@
|
||||||
|
// Copyright 2022 The Gitea Authors. All rights reserved.
|
||||||
|
// SPDX-License-Identifier: MIT
|
||||||
|
|
||||||
|
package integration
|
||||||
|
|
||||||
|
import (
|
||||||
|
"archive/tar"
|
||||||
|
"archive/zip"
|
||||||
|
"bytes"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"net/http"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"code.gitea.io/gitea/models/db"
|
||||||
|
"code.gitea.io/gitea/models/packages"
|
||||||
|
"code.gitea.io/gitea/models/unittest"
|
||||||
|
user_model "code.gitea.io/gitea/models/user"
|
||||||
|
conda_module "code.gitea.io/gitea/modules/packages/conda"
|
||||||
|
"code.gitea.io/gitea/tests"
|
||||||
|
|
||||||
|
"github.com/dsnet/compress/bzip2"
|
||||||
|
"github.com/klauspost/compress/zstd"
|
||||||
|
"github.com/stretchr/testify/assert"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestPackageConda(t *testing.T) {
|
||||||
|
defer tests.PrepareTestEnv(t)()
|
||||||
|
|
||||||
|
user := unittest.AssertExistsAndLoadBean(t, &user_model.User{ID: 2})
|
||||||
|
|
||||||
|
packageName := "test_package"
|
||||||
|
packageVersion := "1.0.1"
|
||||||
|
|
||||||
|
channel := "test-channel"
|
||||||
|
root := fmt.Sprintf("/api/packages/%s/conda", user.Name)
|
||||||
|
|
||||||
|
t.Run("Upload", func(t *testing.T) {
|
||||||
|
tarContent := func() []byte {
|
||||||
|
var buf bytes.Buffer
|
||||||
|
tw := tar.NewWriter(&buf)
|
||||||
|
|
||||||
|
content := []byte(`{"name":"` + packageName + `","version":"` + packageVersion + `","subdir":"noarch","build":"xxx"}`)
|
||||||
|
|
||||||
|
hdr := &tar.Header{
|
||||||
|
Name: "info/index.json",
|
||||||
|
Mode: 0o600,
|
||||||
|
Size: int64(len(content)),
|
||||||
|
}
|
||||||
|
tw.WriteHeader(hdr)
|
||||||
|
tw.Write(content)
|
||||||
|
tw.Close()
|
||||||
|
return buf.Bytes()
|
||||||
|
}()
|
||||||
|
|
||||||
|
t.Run(".tar.bz2", func(t *testing.T) {
|
||||||
|
defer tests.PrintCurrentTest(t)()
|
||||||
|
|
||||||
|
var buf bytes.Buffer
|
||||||
|
bw, _ := bzip2.NewWriter(&buf, nil)
|
||||||
|
io.Copy(bw, bytes.NewReader(tarContent))
|
||||||
|
bw.Close()
|
||||||
|
|
||||||
|
filename := fmt.Sprintf("%s-%s.tar.bz2", packageName, packageVersion)
|
||||||
|
|
||||||
|
req := NewRequestWithBody(t, "PUT", root+"/"+filename, bytes.NewReader(buf.Bytes()))
|
||||||
|
MakeRequest(t, req, http.StatusUnauthorized)
|
||||||
|
|
||||||
|
req = NewRequestWithBody(t, "PUT", root+"/"+filename, bytes.NewReader(buf.Bytes()))
|
||||||
|
AddBasicAuthHeader(req, user.Name)
|
||||||
|
MakeRequest(t, req, http.StatusCreated)
|
||||||
|
|
||||||
|
req = NewRequestWithBody(t, "PUT", root+"/"+filename, bytes.NewReader(buf.Bytes()))
|
||||||
|
AddBasicAuthHeader(req, user.Name)
|
||||||
|
MakeRequest(t, req, http.StatusConflict)
|
||||||
|
|
||||||
|
pvs, err := packages.GetVersionsByPackageType(db.DefaultContext, user.ID, packages.TypeConda)
|
||||||
|
assert.NoError(t, err)
|
||||||
|
assert.Len(t, pvs, 1)
|
||||||
|
|
||||||
|
pd, err := packages.GetPackageDescriptor(db.DefaultContext, pvs[0])
|
||||||
|
assert.NoError(t, err)
|
||||||
|
assert.Nil(t, pd.SemVer)
|
||||||
|
assert.IsType(t, &conda_module.VersionMetadata{}, pd.Metadata)
|
||||||
|
assert.Equal(t, packageName, pd.Package.Name)
|
||||||
|
assert.Equal(t, packageVersion, pd.Version.Version)
|
||||||
|
assert.Empty(t, pd.PackageProperties.GetByName(conda_module.PropertyChannel))
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run(".conda", func(t *testing.T) {
|
||||||
|
defer tests.PrintCurrentTest(t)()
|
||||||
|
|
||||||
|
var infoBuf bytes.Buffer
|
||||||
|
zsw, _ := zstd.NewWriter(&infoBuf)
|
||||||
|
io.Copy(zsw, bytes.NewReader(tarContent))
|
||||||
|
zsw.Close()
|
||||||
|
|
||||||
|
var buf bytes.Buffer
|
||||||
|
zpw := zip.NewWriter(&buf)
|
||||||
|
w, _ := zpw.Create("info-x.tar.zst")
|
||||||
|
w.Write(infoBuf.Bytes())
|
||||||
|
zpw.Close()
|
||||||
|
|
||||||
|
fullName := channel + "/" + packageName
|
||||||
|
filename := fmt.Sprintf("%s-%s.conda", packageName, packageVersion)
|
||||||
|
|
||||||
|
req := NewRequestWithBody(t, "PUT", root+"/"+channel+"/"+filename, bytes.NewReader(buf.Bytes()))
|
||||||
|
MakeRequest(t, req, http.StatusUnauthorized)
|
||||||
|
|
||||||
|
req = NewRequestWithBody(t, "PUT", root+"/"+channel+"/"+filename, bytes.NewReader(buf.Bytes()))
|
||||||
|
AddBasicAuthHeader(req, user.Name)
|
||||||
|
MakeRequest(t, req, http.StatusCreated)
|
||||||
|
|
||||||
|
req = NewRequestWithBody(t, "PUT", root+"/"+channel+"/"+filename, bytes.NewReader(buf.Bytes()))
|
||||||
|
AddBasicAuthHeader(req, user.Name)
|
||||||
|
MakeRequest(t, req, http.StatusConflict)
|
||||||
|
|
||||||
|
pvs, err := packages.GetVersionsByPackageType(db.DefaultContext, user.ID, packages.TypeConda)
|
||||||
|
assert.NoError(t, err)
|
||||||
|
assert.Len(t, pvs, 2)
|
||||||
|
|
||||||
|
pds, err := packages.GetPackageDescriptors(db.DefaultContext, pvs)
|
||||||
|
assert.NoError(t, err)
|
||||||
|
|
||||||
|
assert.Condition(t, func() bool {
|
||||||
|
for _, pd := range pds {
|
||||||
|
if pd.Package.Name == fullName {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
})
|
||||||
|
|
||||||
|
for _, pd := range pds {
|
||||||
|
if pd.Package.Name == fullName {
|
||||||
|
assert.Nil(t, pd.SemVer)
|
||||||
|
assert.IsType(t, &conda_module.VersionMetadata{}, pd.Metadata)
|
||||||
|
assert.Equal(t, fullName, pd.Package.Name)
|
||||||
|
assert.Equal(t, packageVersion, pd.Version.Version)
|
||||||
|
assert.Equal(t, channel, pd.PackageProperties.GetByName(conda_module.PropertyChannel))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("Download", func(t *testing.T) {
|
||||||
|
t.Run(".tar.bz2", func(t *testing.T) {
|
||||||
|
defer tests.PrintCurrentTest(t)()
|
||||||
|
|
||||||
|
req := NewRequest(t, "GET", fmt.Sprintf("%s/noarch/%s-%s-xxx.tar.bz2", root, packageName, packageVersion))
|
||||||
|
MakeRequest(t, req, http.StatusOK)
|
||||||
|
|
||||||
|
req = NewRequest(t, "GET", fmt.Sprintf("%s/%s/noarch/%s-%s-xxx.tar.bz2", root, channel, packageName, packageVersion))
|
||||||
|
MakeRequest(t, req, http.StatusNotFound)
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run(".conda", func(t *testing.T) {
|
||||||
|
defer tests.PrintCurrentTest(t)()
|
||||||
|
|
||||||
|
req := NewRequest(t, "GET", fmt.Sprintf("%s/noarch/%s-%s-xxx.conda", root, packageName, packageVersion))
|
||||||
|
MakeRequest(t, req, http.StatusNotFound)
|
||||||
|
|
||||||
|
req = NewRequest(t, "GET", fmt.Sprintf("%s/%s/noarch/%s-%s-xxx.conda", root, channel, packageName, packageVersion))
|
||||||
|
MakeRequest(t, req, http.StatusOK)
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("EnumeratePackages", func(t *testing.T) {
|
||||||
|
type Info struct {
|
||||||
|
Subdir string `json:"subdir"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type PackageInfo struct {
|
||||||
|
Name string `json:"name"`
|
||||||
|
Version string `json:"version"`
|
||||||
|
NoArch string `json:"noarch"`
|
||||||
|
Subdir string `json:"subdir"`
|
||||||
|
Timestamp int64 `json:"timestamp"`
|
||||||
|
Build string `json:"build"`
|
||||||
|
BuildNumber int64 `json:"build_number"`
|
||||||
|
Dependencies []string `json:"depends"`
|
||||||
|
License string `json:"license"`
|
||||||
|
LicenseFamily string `json:"license_family"`
|
||||||
|
HashMD5 string `json:"md5"`
|
||||||
|
HashSHA256 string `json:"sha256"`
|
||||||
|
Size int64 `json:"size"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type RepoData struct {
|
||||||
|
Info Info `json:"info"`
|
||||||
|
Packages map[string]*PackageInfo `json:"packages"`
|
||||||
|
PackagesConda map[string]*PackageInfo `json:"packages.conda"`
|
||||||
|
Removed map[string]*PackageInfo `json:"removed"`
|
||||||
|
}
|
||||||
|
|
||||||
|
req := NewRequest(t, "GET", fmt.Sprintf("%s/noarch/repodata.json", root))
|
||||||
|
resp := MakeRequest(t, req, http.StatusOK)
|
||||||
|
assert.Equal(t, "application/json", resp.Header().Get("Content-Type"))
|
||||||
|
|
||||||
|
req = NewRequest(t, "GET", fmt.Sprintf("%s/noarch/repodata.json.bz2", root))
|
||||||
|
resp = MakeRequest(t, req, http.StatusOK)
|
||||||
|
assert.Equal(t, "application/x-bzip2", resp.Header().Get("Content-Type"))
|
||||||
|
|
||||||
|
req = NewRequest(t, "GET", fmt.Sprintf("%s/noarch/current_repodata.json", root))
|
||||||
|
resp = MakeRequest(t, req, http.StatusOK)
|
||||||
|
assert.Equal(t, "application/json", resp.Header().Get("Content-Type"))
|
||||||
|
|
||||||
|
req = NewRequest(t, "GET", fmt.Sprintf("%s/noarch/current_repodata.json.bz2", root))
|
||||||
|
resp = MakeRequest(t, req, http.StatusOK)
|
||||||
|
assert.Equal(t, "application/x-bzip2", resp.Header().Get("Content-Type"))
|
||||||
|
|
||||||
|
t.Run(".tar.bz2", func(t *testing.T) {
|
||||||
|
defer tests.PrintCurrentTest(t)()
|
||||||
|
|
||||||
|
pv, err := packages.GetVersionByNameAndVersion(db.DefaultContext, user.ID, packages.TypeConda, packageName, packageVersion)
|
||||||
|
assert.NoError(t, err)
|
||||||
|
|
||||||
|
pd, err := packages.GetPackageDescriptor(db.DefaultContext, pv)
|
||||||
|
assert.NoError(t, err)
|
||||||
|
|
||||||
|
req := NewRequest(t, "GET", fmt.Sprintf("%s/noarch/repodata.json", root))
|
||||||
|
resp := MakeRequest(t, req, http.StatusOK)
|
||||||
|
|
||||||
|
var result RepoData
|
||||||
|
DecodeJSON(t, resp, &result)
|
||||||
|
|
||||||
|
assert.Equal(t, "noarch", result.Info.Subdir)
|
||||||
|
assert.Empty(t, result.PackagesConda)
|
||||||
|
assert.Empty(t, result.Removed)
|
||||||
|
|
||||||
|
filename := fmt.Sprintf("%s-%s-xxx.tar.bz2", packageName, packageVersion)
|
||||||
|
assert.Contains(t, result.Packages, filename)
|
||||||
|
packageInfo := result.Packages[filename]
|
||||||
|
assert.Equal(t, packageName, packageInfo.Name)
|
||||||
|
assert.Equal(t, packageVersion, packageInfo.Version)
|
||||||
|
assert.Equal(t, "noarch", packageInfo.Subdir)
|
||||||
|
assert.Equal(t, "xxx", packageInfo.Build)
|
||||||
|
assert.Equal(t, pd.Files[0].Blob.HashMD5, packageInfo.HashMD5)
|
||||||
|
assert.Equal(t, pd.Files[0].Blob.HashSHA256, packageInfo.HashSHA256)
|
||||||
|
assert.Equal(t, pd.Files[0].Blob.Size, packageInfo.Size)
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run(".conda", func(t *testing.T) {
|
||||||
|
defer tests.PrintCurrentTest(t)()
|
||||||
|
|
||||||
|
pv, err := packages.GetVersionByNameAndVersion(db.DefaultContext, user.ID, packages.TypeConda, channel+"/"+packageName, packageVersion)
|
||||||
|
assert.NoError(t, err)
|
||||||
|
|
||||||
|
pd, err := packages.GetPackageDescriptor(db.DefaultContext, pv)
|
||||||
|
assert.NoError(t, err)
|
||||||
|
|
||||||
|
req := NewRequest(t, "GET", fmt.Sprintf("%s/%s/noarch/repodata.json", root, channel))
|
||||||
|
resp := MakeRequest(t, req, http.StatusOK)
|
||||||
|
|
||||||
|
var result RepoData
|
||||||
|
DecodeJSON(t, resp, &result)
|
||||||
|
|
||||||
|
assert.Equal(t, "noarch", result.Info.Subdir)
|
||||||
|
assert.Empty(t, result.Packages)
|
||||||
|
assert.Empty(t, result.Removed)
|
||||||
|
|
||||||
|
filename := fmt.Sprintf("%s-%s-xxx.conda", packageName, packageVersion)
|
||||||
|
assert.Contains(t, result.PackagesConda, filename)
|
||||||
|
packageInfo := result.PackagesConda[filename]
|
||||||
|
assert.Equal(t, packageName, packageInfo.Name)
|
||||||
|
assert.Equal(t, packageVersion, packageInfo.Version)
|
||||||
|
assert.Equal(t, "noarch", packageInfo.Subdir)
|
||||||
|
assert.Equal(t, "xxx", packageInfo.Build)
|
||||||
|
assert.Equal(t, pd.Files[0].Blob.HashMD5, packageInfo.HashMD5)
|
||||||
|
assert.Equal(t, pd.Files[0].Blob.HashSHA256, packageInfo.HashSHA256)
|
||||||
|
assert.Equal(t, pd.Files[0].Blob.Size, packageInfo.Size)
|
||||||
|
})
|
||||||
|
})
|
||||||
|
}
|
20
web_src/svg/gitea-conda.svg
Normal file
20
web_src/svg/gitea-conda.svg
Normal file
|
@ -0,0 +1,20 @@
|
||||||
|
<?xml version="1.0" encoding="UTF-8"?>
|
||||||
|
<svg viewBox="0 0 32 32" xmlns="http://www.w3.org/2000/svg">
|
||||||
|
<path d="M16.559,8.137a7.2,7.2,0,0,0-1.234-1.708,7.586,7.586,0,0,0-.19,2.183,5.161,5.161,0,0,1,1.424-.475Z" fill="#43b02a" fill-rule="evenodd" stroke="#43b02a" stroke-width=".068489px"/>
|
||||||
|
<path d="M13.617,9.466a7.992,7.992,0,0,0-1.993-1.2,8.123,8.123,0,0,0,.885,2.183c0,.063.443-.475,1.108-.981Z" fill="#43b02a" fill-rule="evenodd" stroke="#43b02a" stroke-width=".068489px"/>
|
||||||
|
<path d="M17.445,7.188a9.143,9.143,0,0,1,1.3-2.246A7.585,7.585,0,0,0,17,2.854a8.35,8.35,0,0,0-1.3,2.278,8.451,8.451,0,0,1,1.74,2.056Z" fill="#43b02a" fill-rule="evenodd" stroke="#43b02a" stroke-width=".068489px"/>
|
||||||
|
<path d="m11.592 11.744a10.276 10.276 0 0 0-2.692-0.158 7.478 7.478 0 0 0 1.93 1.9 6.858 6.858 0 0 1 0.759-1.74z" fill="#43b02a" fill-rule="evenodd" stroke="#43b02a" stroke-width=".068489px"/>
|
||||||
|
<path d="m6.878 15.161a7.44 7.44 0 0 1 2.942-1.139 10.019 10.019 0 0 1-2.056-2.278 7.639 7.639 0 0 0-2.847 1.2 7.11 7.11 0 0 0 1.961 2.215z" fill="#43b02a" fill-rule="evenodd" stroke="#43b02a" stroke-width=".068489px"/>
|
||||||
|
<path d="m10.516 14.876a6.16 6.16 0 0 0-2.815 0.886 9.936 9.936 0 0 0 2.815 1.2 7.683 7.683 0 0 1 0-2.088z" fill="#43b02a" fill-rule="evenodd" stroke="#43b02a" stroke-width=".068489px"/>
|
||||||
|
<path d="M14.281,5.543A7.839,7.839,0,0,0,11.592,4.4,8.361,8.361,0,0,0,11.4,7,8.875,8.875,0,0,1,13.87,8.264a10.292,10.292,0,0,1,.411-2.721Z" fill="#43b02a" fill-rule="evenodd" stroke="#43b02a" stroke-width=".068489px"/>
|
||||||
|
<path d="M24.025,3.234a20.488,20.488,0,0,1,.917,4.112,6.823,6.823,0,0,0-3.068,1.519,7.443,7.443,0,0,1,1.55,1.044,1.351,1.351,0,0,0,1.645.316,36.938,36.938,0,0,0,2.721-2.72,1.273,1.273,0,0,0-.159-1.835,20.521,20.521,0,0,0-3.606-2.436Z" fill="#43b02a" fill-rule="evenodd" stroke="#43b02a" stroke-width=".068489px"/>
|
||||||
|
<path d="M4.379,12.06A8.67,8.67,0,0,1,7.226,10.8a7.763,7.763,0,0,1-.759-2.974A14.687,14.687,0,0,0,4.379,12.06Z" fill="#43b02a" fill-rule="evenodd" stroke="#43b02a" stroke-width=".068489px"/>
|
||||||
|
<path d="M11.339,10.668a9.991,9.991,0,0,1-.949-2.784,7.928,7.928,0,0,0-2.911-.126,7.312,7.312,0,0,0,.791,2.879,9.664,9.664,0,0,1,3.069.031Z" fill="#43b02a" fill-rule="evenodd" stroke="#43b02a" stroke-width=".068489px"/>
|
||||||
|
<path d="M6.119,15.73a8.894,8.894,0,0,1-2.025-2.373,14.208,14.208,0,0,0-.063,4.9A8.522,8.522,0,0,1,6.119,15.73Z" fill="#43b02a" fill-rule="evenodd" stroke="#43b02a" stroke-width=".068489px"/>
|
||||||
|
<path d="M22.538,3.487A7.581,7.581,0,0,0,20.323,5.1a11.789,11.789,0,0,1,.823,2.5,9.775,9.775,0,0,1,2.309-1.329,6.593,6.593,0,0,0-.917-2.784Z" fill="#43b02a" fill-rule="evenodd" stroke="#43b02a" stroke-width=".068489px"/>
|
||||||
|
<path d="M19.374,6.3a8.608,8.608,0,0,0-.822,1.676h0a9.645,9.645,0,0,1,1.329.19A7.568,7.568,0,0,0,19.374,6.3Z" fill="#43b02a" fill-rule="evenodd" stroke="#43b02a" stroke-width=".068489px"/>
|
||||||
|
<path d="M19.659,3.9a9.577,9.577,0,0,1,2.056-1.487A15.38,15.38,0,0,0,18.046,2a9.709,9.709,0,0,1,1.613,1.9Z" fill="#43b02a" fill-rule="evenodd" stroke="#43b02a" stroke-width=".068489px"/>
|
||||||
|
<path d="M27.378,23.892c-1.993-1.9-2.4-3.132-4.081-1.835a7.837,7.837,0,0,1-12.591-4.144A10.179,10.179,0,0,1,6.878,16.3a9.427,9.427,0,0,0-2.562,3.321H4.284C7.163,30.5,21.178,33.035,27.663,26.233,28.739,25.094,27.758,24.3,27.378,23.892ZM6.309,20.855a7.559,7.559,0,0,1,.917-2.025,6.872,6.872,0,0,0,2.151.538c1.013,2.689,4.556,6.264,8.922,6.264a9.632,9.632,0,0,0,6.3-2.309,12.841,12.841,0,0,1,1.772,1.771c.095.127.095.159.095.159C20.7,30.283,10.928,29.555,6.309,20.855Z" fill="#43b02a"/>
|
||||||
|
<path d="M10.67,4.11a19.934,19.934,0,0,0-.214,2.509,10.512,10.512,0,0,0-2.689-.093A18,18,0,0,1,10.67,4.11Z" fill="#43b02a" fill-rule="evenodd" stroke="#43b02a" stroke-width=".066605px"/>
|
||||||
|
<path d="m12.26 3.274a9.107 9.107 0 0 1 2.445 1.053 14.083 14.083 0 0 1 1.253-2.137 12.106 12.106 0 0 0-3.698 1.084z" fill="#43b02a" fill-rule="evenodd" stroke="#43b02a" stroke-width=".066605px"/>
|
||||||
|
</svg>
|
After Width: | Height: | Size: 3.8 KiB |
Loading…
Reference in a new issue