henry99a commited on
Commit
ca7217f
·
0 Parent(s):

Clean commit for Hugging Face Spaces without binary files

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. .github/CODE_OF_CONDUCT.md +128 -0
  2. .github/CONTRIBUTING.md +11 -0
  3. .github/FUNDING.yml +13 -0
  4. .github/ISSUE_TEMPLATE/feature_request.yml +25 -0
  5. .github/SECURITY.md +29 -0
  6. .github/workflows/codeql-analysis.yml +41 -0
  7. .github/workflows/docker.yml +73 -0
  8. .github/workflows/linter.yml +28 -0
  9. .github/workflows/release.yml +49 -0
  10. .github/workflows/stale.yml +16 -0
  11. .github/workflows/test.yml +36 -0
  12. .gitignore +28 -0
  13. .golangci.yaml +33 -0
  14. Dockerfile +32 -0
  15. LICENSE +201 -0
  16. Makefile +132 -0
  17. README.md +73 -0
  18. cmd/cmd.go +111 -0
  19. cmd/init.go +9 -0
  20. cmd/server/main.go +35 -0
  21. collection/maps/cimap.go +96 -0
  22. collection/maps/cimap_test.go +90 -0
  23. collection/maps/orderedmap.go +79 -0
  24. collection/maps/orderedmap_test.go +112 -0
  25. collection/sets/orderedset.go +76 -0
  26. collection/sets/orderedset_test.go +32 -0
  27. collection/slices/flatten.go +18 -0
  28. collection/slices/flatten_test.go +16 -0
  29. collection/slices/wslice.go +48 -0
  30. collection/slices/wslice_test.go +28 -0
  31. collection/unionfind/README.md +3 -0
  32. collection/unionfind/unionfind.go +235 -0
  33. collection/unionfind/unionfind_test.go +374 -0
  34. common/bufferpool/bufferpool.go +29 -0
  35. common/bufferpool/bufferpool_test.go +21 -0
  36. common/cluster/group.go +42 -0
  37. common/cluster/group_test.go +56 -0
  38. common/cluster/locatable.go +22 -0
  39. common/cluster/sort.go +44 -0
  40. common/comparer/compare.go +16 -0
  41. common/comparer/compare_test.go +28 -0
  42. common/convertor/convert.go +11 -0
  43. common/convertor/convert_test.go +22 -0
  44. common/convertor/replace.go +18 -0
  45. common/convertor/replace_test.go +19 -0
  46. common/fetch/body.go +25 -0
  47. common/fetch/fetch.go +167 -0
  48. common/fetch/option.go +92 -0
  49. common/js/js.go +30 -0
  50. common/js/js_test.go +48 -0
.github/CODE_OF_CONDUCT.md ADDED
@@ -0,0 +1,128 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Contributor Covenant Code of Conduct
2
+
3
+ ## Our Pledge
4
+
5
+ We as members, contributors, and leaders pledge to make participation in our
6
+ community a harassment-free experience for everyone, regardless of age, body
7
+ size, visible or invisible disability, ethnicity, sex characteristics, gender
8
+ identity and expression, level of experience, education, socio-economic status,
9
+ nationality, personal appearance, race, religion, or sexual identity
10
+ and orientation.
11
+
12
+ We pledge to act and interact in ways that contribute to an open, welcoming,
13
+ diverse, inclusive, and healthy community.
14
+
15
+ ## Our Standards
16
+
17
+ Examples of behavior that contributes to a positive environment for our
18
+ community include:
19
+
20
+ * Demonstrating empathy and kindness toward other people
21
+ * Being respectful of differing opinions, viewpoints, and experiences
22
+ * Giving and gracefully accepting constructive feedback
23
+ * Accepting responsibility and apologizing to those affected by our mistakes,
24
+ and learning from the experience
25
+ * Focusing on what is best not just for us as individuals, but for the
26
+ overall community
27
+
28
+ Examples of unacceptable behavior include:
29
+
30
+ * The use of sexualized language or imagery, and sexual attention or
31
+ advances of any kind
32
+ * Trolling, insulting or derogatory comments, and personal or political attacks
33
+ * Public or private harassment
34
+ * Publishing others' private information, such as a physical or email
35
+ address, without their explicit permission
36
+ * Other conduct which could reasonably be considered inappropriate in a
37
+ professional setting
38
+
39
+ ## Enforcement Responsibilities
40
+
41
+ Community leaders are responsible for clarifying and enforcing our standards of
42
+ acceptable behavior and will take appropriate and fair corrective action in
43
+ response to any behavior that they deem inappropriate, threatening, offensive,
44
+ or harmful.
45
+
46
+ Community leaders have the right and responsibility to remove, edit, or reject
47
+ comments, commits, code, wiki edits, issues, and other contributions that are
48
+ not aligned to this Code of Conduct, and will communicate reasons for moderation
49
+ decisions when appropriate.
50
+
51
+ ## Scope
52
+
53
+ This Code of Conduct applies within all community spaces, and also applies when
54
+ an individual is officially representing the community in public spaces.
55
+ Examples of representing our community include using an official e-mail address,
56
+ posting via an official social media account, or acting as an appointed
57
+ representative at an online or offline event.
58
+
59
+ ## Enforcement
60
+
61
+ Instances of abusive, harassing, or otherwise unacceptable behavior may be
62
+ reported to the community leaders responsible for enforcement at
63
+ xjasonlyu@gmail.com.
64
+ All complaints will be reviewed and investigated promptly and fairly.
65
+
66
+ All community leaders are obligated to respect the privacy and security of the
67
+ reporter of any incident.
68
+
69
+ ## Enforcement Guidelines
70
+
71
+ Community leaders will follow these Community Impact Guidelines in determining
72
+ the consequences for any action they deem in violation of this Code of Conduct:
73
+
74
+ ### 1. Correction
75
+
76
+ **Community Impact**: Use of inappropriate language or other behavior deemed
77
+ unprofessional or unwelcome in the community.
78
+
79
+ **Consequence**: A private, written warning from community leaders, providing
80
+ clarity around the nature of the violation and an explanation of why the
81
+ behavior was inappropriate. A public apology may be requested.
82
+
83
+ ### 2. Warning
84
+
85
+ **Community Impact**: A violation through a single incident or series
86
+ of actions.
87
+
88
+ **Consequence**: A warning with consequences for continued behavior. No
89
+ interaction with the people involved, including unsolicited interaction with
90
+ those enforcing the Code of Conduct, for a specified period of time. This
91
+ includes avoiding interactions in community spaces as well as external channels
92
+ like social media. Violating these terms may lead to a temporary or
93
+ permanent ban.
94
+
95
+ ### 3. Temporary Ban
96
+
97
+ **Community Impact**: A serious violation of community standards, including
98
+ sustained inappropriate behavior.
99
+
100
+ **Consequence**: A temporary ban from any sort of interaction or public
101
+ communication with the community for a specified period of time. No public or
102
+ private interaction with the people involved, including unsolicited interaction
103
+ with those enforcing the Code of Conduct, is allowed during this period.
104
+ Violating these terms may lead to a permanent ban.
105
+
106
+ ### 4. Permanent Ban
107
+
108
+ **Community Impact**: Demonstrating a pattern of violation of community
109
+ standards, including sustained inappropriate behavior, harassment of an
110
+ individual, or aggression toward or disparagement of classes of individuals.
111
+
112
+ **Consequence**: A permanent ban from any sort of public interaction within
113
+ the community.
114
+
115
+ ## Attribution
116
+
117
+ This Code of Conduct is adapted from the [Contributor Covenant][homepage],
118
+ version 2.0, available at
119
+ https://www.contributor-covenant.org/version/2/0/code_of_conduct.html.
120
+
121
+ Community Impact Guidelines were inspired by [Mozilla's code of conduct
122
+ enforcement ladder](https://github.com/mozilla/diversity).
123
+
124
+ [homepage]: https://www.contributor-covenant.org
125
+
126
+ For answers to common questions about this code of conduct, see the FAQ at
127
+ https://www.contributor-covenant.org/faq. Translations are available at
128
+ https://www.contributor-covenant.org/translations.
.github/CONTRIBUTING.md ADDED
@@ -0,0 +1,11 @@
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ## Contributing
2
+
3
+ - With issues:
4
+ - Use the search tool before opening a new issue.
5
+ - Please provide source code and commit sha if you found a bug.
6
+ - Review existing issues and provide feedback or react to them.
7
+
8
+ - With pull requests:
9
+ - Open your pull request against `main`
10
+ - It should pass all tests in the available continuous integration systems such as GitHub Actions.
11
+ - You should add/modify tests to cover your proposed code changes.
.github/FUNDING.yml ADDED
@@ -0,0 +1,13 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # These are supported funding model platforms
2
+
3
+ github: [xjasonlyu]
4
+ patreon: # Replace with a single Patreon username
5
+ open_collective: # Replace with a single Open Collective username
6
+ ko_fi: # Replace with a single Ko-fi username
7
+ tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
8
+ community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry
9
+ liberapay: # Replace with a single Liberapay username
10
+ issuehunt: # Replace with a single IssueHunt username
11
+ otechie: # Replace with a single Otechie username
12
+ lfx_crowdfunding: # Replace with a single LFX Crowdfunding project-name e.g., cloud-foundry
13
+ custom: # Replace with up to 4 custom sponsorship URLs e.g., ['link1', 'link2']
.github/ISSUE_TEMPLATE/feature_request.yml ADDED
@@ -0,0 +1,25 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ name: Feature request
2
+ description: Suggest an idea or improvement
3
+ title: "[Feature] "
4
+ body:
5
+ - type: textarea
6
+ id: description
7
+ attributes:
8
+ label: Description
9
+ placeholder: A clear description of the feature or enhancement.
10
+ validations:
11
+ required: true
12
+
13
+ - type: textarea
14
+ id: related
15
+ attributes:
16
+ label: Is this feature related to a specific bug?
17
+ description: Please include a bug references if yes.
18
+
19
+ - type: textarea
20
+ id: solution
21
+ attributes:
22
+ label: Do you have a specific solution in mind?
23
+ description: >
24
+ Please include any details about a solution that you have in mind,
25
+ including any alternatives considered.
.github/SECURITY.md ADDED
@@ -0,0 +1,29 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Security Policy
2
+
3
+ ## Supported Versions
4
+
5
+ | Version | Supported |
6
+ | :-----: | :----------------: |
7
+ | 1.2.x | :white_check_mark: |
8
+ | 1.1.x | :x: |
9
+ | 1.0.x | :x: |
10
+
11
+ ## Reporting a Vulnerability
12
+
13
+ If you believe you have found a security vulnerability in this repository, please report it to me through coordinated disclosure.
14
+
15
+ **Please do not report security vulnerabilities through public GitHub issues, discussions, or pull requests.**
16
+
17
+ Instead, please send an email to xjasonlyu[@]gmail.com.
18
+
19
+ Please include as much of the information listed below as you can to help me better understand and resolve the issue:
20
+
21
+ * The type of issue (e.g., buffer overflow, payload attack)
22
+ * Full paths of source file(s) related to the manifestation of the issue
23
+ * The location of the affected source code (tag/branch/commit or direct URL)
24
+ * Any special configuration required to reproduce the issue
25
+ * Step-by-step instructions to reproduce the issue
26
+ * Proof-of-concept or exploit code (if possible)
27
+ * Impact of the issue, including how an attacker might exploit the issue
28
+
29
+ This information will help me triage your report more quickly.
.github/workflows/codeql-analysis.yml ADDED
@@ -0,0 +1,41 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ name: "CodeQL"
2
+
3
+ concurrency:
4
+ group: codeql-${{ github.event_name }}-${{ github.ref }}
5
+ cancel-in-progress: true
6
+
7
+ on:
8
+ push:
9
+ branches: [ main ]
10
+ pull_request:
11
+
12
+ jobs:
13
+ analyze:
14
+ name: Analyze
15
+ runs-on: ubuntu-latest
16
+
17
+ strategy:
18
+ fail-fast: false
19
+ matrix:
20
+ language: [ 'go' ]
21
+
22
+ steps:
23
+ - name: Checkout repository
24
+ uses: actions/checkout@v5
25
+
26
+ - name: Setup Go
27
+ uses: actions/setup-go@v6
28
+ with:
29
+ check-latest: true
30
+ go-version-file: 'go.mod'
31
+
32
+ - name: Initialize CodeQL
33
+ uses: github/codeql-action/init@v4
34
+ with:
35
+ languages: ${{ matrix.language }}
36
+
37
+ - name: Autobuild
38
+ uses: github/codeql-action/autobuild@v4
39
+
40
+ - name: Perform CodeQL Analysis
41
+ uses: github/codeql-action/analyze@v4
.github/workflows/docker.yml ADDED
@@ -0,0 +1,73 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ name: Publish Docker Image
2
+
3
+ concurrency:
4
+ group: docker-${{ github.event_name }}-${{ github.ref }}
5
+ cancel-in-progress: true
6
+
7
+ on:
8
+ push:
9
+ branches:
10
+ - 'main'
11
+ tags:
12
+ - '*'
13
+
14
+ jobs:
15
+
16
+ build:
17
+ name: Build
18
+ runs-on: ubuntu-latest
19
+ steps:
20
+
21
+ - name: Check out code into the Go module directory
22
+ uses: actions/checkout@v5
23
+ with:
24
+ fetch-depth: 0
25
+
26
+ - name: Set up QEMU
27
+ uses: docker/setup-qemu-action@v3
28
+ with:
29
+ platforms: all
30
+
31
+ - name: Set up Docker Buildx
32
+ uses: docker/setup-buildx-action@v3
33
+ with:
34
+ version: latest
35
+
36
+ - name: Login to GitHub Container Registry
37
+ uses: docker/login-action@v3
38
+ with:
39
+ registry: ghcr.io
40
+ username: ${{ github.repository_owner }}
41
+ password: ${{ secrets.GITHUB_TOKEN }}
42
+
43
+ - name: Get Version
44
+ id: shell
45
+ run: |
46
+ echo "version=$(git describe --abbrev=0 --tags HEAD | cut -d'v' -f 2)" >> $GITHUB_OUTPUT
47
+
48
+ - name: Build and Push (dev)
49
+ if: github.ref == 'refs/heads/main'
50
+ uses: docker/build-push-action@v6
51
+ with:
52
+ context: .
53
+ file: Dockerfile
54
+ push: true
55
+ platforms: linux/amd64,linux/arm64
56
+ tags: |
57
+ ghcr.io/metatube-community/metatube-server:dev
58
+ cache-from: type=gha
59
+ cache-to: type=gha,mode=max
60
+
61
+ - name: Build and Push (latest)
62
+ if: startsWith(github.ref, 'refs/tags/')
63
+ uses: docker/build-push-action@v6
64
+ with:
65
+ context: .
66
+ file: Dockerfile
67
+ push: true
68
+ platforms: linux/amd64,linux/arm64
69
+ tags: |
70
+ ghcr.io/metatube-community/metatube-server:latest
71
+ ghcr.io/metatube-community/metatube-server:${{ steps.shell.outputs.version }}
72
+ cache-from: type=gha
73
+ cache-to: type=gha,mode=max
.github/workflows/linter.yml ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ name: Linter
2
+
3
+ concurrency:
4
+ group: linter-${{ github.event_name }}-${{ github.ref }}
5
+ cancel-in-progress: true
6
+
7
+ on:
8
+ push:
9
+ branches:
10
+ - 'main'
11
+ pull_request:
12
+
13
+ jobs:
14
+ golangci-lint:
15
+ runs-on: ubuntu-latest
16
+ steps:
17
+ - uses: actions/checkout@v5
18
+
19
+ - name: Setup Go
20
+ uses: actions/setup-go@v6
21
+ with:
22
+ check-latest: true
23
+ go-version-file: 'go.mod'
24
+
25
+ - name: golangci-lint
26
+ uses: golangci/golangci-lint-action@v8
27
+ with:
28
+ version: latest
.github/workflows/release.yml ADDED
@@ -0,0 +1,49 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ name: Publish Go Releases
2
+
3
+ concurrency:
4
+ group: release-${{ github.event_name }}-${{ github.ref }}
5
+ cancel-in-progress: true
6
+
7
+ on:
8
+ push:
9
+ tags:
10
+ - '*'
11
+
12
+ jobs:
13
+ build:
14
+ name: Build
15
+ runs-on: ubuntu-latest
16
+ steps:
17
+ - name: Check out code into the Go module directory
18
+ uses: actions/checkout@v5
19
+ with:
20
+ fetch-depth: 0
21
+
22
+ - name: Setup Go
23
+ uses: actions/setup-go@v6
24
+ with:
25
+ check-latest: true
26
+ go-version-file: 'go.mod'
27
+
28
+ - name: Cache go module
29
+ uses: actions/cache@v4
30
+ with:
31
+ path: ~/go/pkg/mod
32
+ key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
33
+ restore-keys: |
34
+ ${{ runner.os }}-go-
35
+
36
+ - name: Build
37
+ if: startsWith(github.ref, 'refs/tags/')
38
+ run: make releases
39
+
40
+ - name: Upload Releases
41
+ uses: softprops/action-gh-release@v2
42
+ if: startsWith(github.ref, 'refs/tags/')
43
+ with:
44
+ body: _Auto Released by Actions_
45
+ files: build/*
46
+ draft: false
47
+ prerelease: false
48
+ repository: metatube-community/metatube-server-releases
49
+ token: ${{ secrets.ORG_PAT }}
.github/workflows/stale.yml ADDED
@@ -0,0 +1,16 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ name: Mark stale issues and pull requests
2
+
3
+ on:
4
+ schedule:
5
+ - cron: "0 10 * * *"
6
+
7
+ jobs:
8
+ stale:
9
+ runs-on: ubuntu-latest
10
+ steps:
11
+ - uses: actions/stale@v10
12
+ with:
13
+ stale-issue-message: 'This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 7 days'
14
+ exempt-issue-labels: 'question,bug,enhancement'
15
+ days-before-stale: 60
16
+ days-before-close: 7
.github/workflows/test.yml ADDED
@@ -0,0 +1,36 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ name: Test
2
+
3
+ concurrency:
4
+ group: test-${{ github.event_name }}-${{ github.ref }}
5
+ cancel-in-progress: true
6
+
7
+ on:
8
+ push:
9
+ branches:
10
+ - 'main'
11
+ pull_request:
12
+
13
+ jobs:
14
+ go-test:
15
+ name: Go Test
16
+ runs-on: ubuntu-latest
17
+ services:
18
+ dind:
19
+ image: docker:dind-rootless
20
+ ports:
21
+ - 2375:2375
22
+
23
+ steps:
24
+ - name: Checkout code
25
+ uses: actions/checkout@v5
26
+
27
+ - name: Setup Go
28
+ uses: actions/setup-go@v6
29
+ with:
30
+ check-latest: true
31
+ go-version-file: 'go.mod'
32
+
33
+ - name: Run test
34
+ run: |
35
+ go version
36
+ go test $(go list ./... | grep -Ev "github.com/metatube-community/metatube-sdk-go/translate")
.gitignore ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # If you prefer the allow list template instead of the deny list, see community template:
2
+ # https://github.com/github/gitignore/blob/main/community/Golang/Go.AllowList.gitignore
3
+ #
4
+ # Binaries for programs and plugins
5
+ *.exe
6
+ *.exe~
7
+ *.dll
8
+ *.so
9
+ *.dylib
10
+
11
+ # Test binary, built with `go test -c`
12
+ *.test
13
+
14
+ # Output of the go coverage tool, specifically when used with LiteIDE
15
+ *.out
16
+
17
+ # Dependency directories (remove the comment below to include it)
18
+ # vendor/
19
+
20
+ # Go workspace file
21
+ go.work
22
+ go.work.sum
23
+
24
+ # env file
25
+ .env
26
+
27
+ # Build files
28
+ build/
.golangci.yaml ADDED
@@ -0,0 +1,33 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ version: "2"
2
+ linters:
3
+ default: none
4
+ enable:
5
+ - govet
6
+ - ineffassign
7
+ - misspell
8
+ - staticcheck
9
+ - unconvert
10
+ - unused
11
+ - usestdlibvars
12
+ exclusions:
13
+ generated: lax
14
+ presets:
15
+ - comments
16
+ - common-false-positives
17
+ - legacy
18
+ - std-error-handling
19
+ paths: [ ]
20
+ formatters:
21
+ enable:
22
+ - gci
23
+ - gofumpt
24
+ settings:
25
+ gci:
26
+ sections:
27
+ - standard
28
+ - default
29
+ - prefix(github.com/metatube-community/metatube-sdk-go)
30
+ custom-order: true
31
+ exclusions:
32
+ generated: lax
33
+ paths: [ ]
Dockerfile ADDED
@@ -0,0 +1,32 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ FROM --platform=$BUILDPLATFORM golang:alpine AS builder
2
+
3
+ ARG TARGETOS
4
+ ARG TARGETARCH
5
+
6
+ WORKDIR /src
7
+ COPY . /src
8
+
9
+ RUN apk add --update --no-cache --no-progress make git \
10
+ && GOOS=$TARGETOS GOARCH=$TARGETARCH make server BUILD_COMMIT=unknown BUILD_VERSION=unknown
11
+
12
+ FROM alpine:latest
13
+ LABEL org.opencontainers.image.licenses=Apache-2.0
14
+ LABEL org.opencontainers.image.source="https://github.com/metatube-community/metatube-sdk-go"
15
+
16
+ COPY --from=builder /src/build/metatube-server .
17
+
18
+ RUN apk add --update --no-cache --no-progress ca-certificates tzdata
19
+
20
+ ENV GIN_MODE=release
21
+ ENV PORT=7860
22
+ ENV TOKEN=""
23
+ ENV DSN=""
24
+ ENV REQUEST_TIMEOUT=""
25
+ ENV DB_MAX_IDLE_CONNS=0
26
+ ENV DB_MAX_OPEN_CONNS=0
27
+ ENV DB_PREPARED_STMT=0
28
+ ENV DB_AUTO_MIGRATE=0
29
+
30
+ EXPOSE 7860
31
+
32
+ ENTRYPOINT ["/metatube-server"]
LICENSE ADDED
@@ -0,0 +1,201 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Apache License
2
+ Version 2.0, January 2004
3
+ http://www.apache.org/licenses/
4
+
5
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
6
+
7
+ 1. Definitions.
8
+
9
+ "License" shall mean the terms and conditions for use, reproduction,
10
+ and distribution as defined by Sections 1 through 9 of this document.
11
+
12
+ "Licensor" shall mean the copyright owner or entity authorized by
13
+ the copyright owner that is granting the License.
14
+
15
+ "Legal Entity" shall mean the union of the acting entity and all
16
+ other entities that control, are controlled by, or are under common
17
+ control with that entity. For the purposes of this definition,
18
+ "control" means (i) the power, direct or indirect, to cause the
19
+ direction or management of such entity, whether by contract or
20
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
21
+ outstanding shares, or (iii) beneficial ownership of such entity.
22
+
23
+ "You" (or "Your") shall mean an individual or Legal Entity
24
+ exercising permissions granted by this License.
25
+
26
+ "Source" form shall mean the preferred form for making modifications,
27
+ including but not limited to software source code, documentation
28
+ source, and configuration files.
29
+
30
+ "Object" form shall mean any form resulting from mechanical
31
+ transformation or translation of a Source form, including but
32
+ not limited to compiled object code, generated documentation,
33
+ and conversions to other media types.
34
+
35
+ "Work" shall mean the work of authorship, whether in Source or
36
+ Object form, made available under the License, as indicated by a
37
+ copyright notice that is included in or attached to the work
38
+ (an example is provided in the Appendix below).
39
+
40
+ "Derivative Works" shall mean any work, whether in Source or Object
41
+ form, that is based on (or derived from) the Work and for which the
42
+ editorial revisions, annotations, elaborations, or other modifications
43
+ represent, as a whole, an original work of authorship. For the purposes
44
+ of this License, Derivative Works shall not include works that remain
45
+ separable from, or merely link (or bind by name) to the interfaces of,
46
+ the Work and Derivative Works thereof.
47
+
48
+ "Contribution" shall mean any work of authorship, including
49
+ the original version of the Work and any modifications or additions
50
+ to that Work or Derivative Works thereof, that is intentionally
51
+ submitted to Licensor for inclusion in the Work by the copyright owner
52
+ or by an individual or Legal Entity authorized to submit on behalf of
53
+ the copyright owner. For the purposes of this definition, "submitted"
54
+ means any form of electronic, verbal, or written communication sent
55
+ to the Licensor or its representatives, including but not limited to
56
+ communication on electronic mailing lists, source code control systems,
57
+ and issue tracking systems that are managed by, or on behalf of, the
58
+ Licensor for the purpose of discussing and improving the Work, but
59
+ excluding communication that is conspicuously marked or otherwise
60
+ designated in writing by the copyright owner as "Not a Contribution."
61
+
62
+ "Contributor" shall mean Licensor and any individual or Legal Entity
63
+ on behalf of whom a Contribution has been received by Licensor and
64
+ subsequently incorporated within the Work.
65
+
66
+ 2. Grant of Copyright License. Subject to the terms and conditions of
67
+ this License, each Contributor hereby grants to You a perpetual,
68
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
69
+ copyright license to reproduce, prepare Derivative Works of,
70
+ publicly display, publicly perform, sublicense, and distribute the
71
+ Work and such Derivative Works in Source or Object form.
72
+
73
+ 3. Grant of Patent License. Subject to the terms and conditions of
74
+ this License, each Contributor hereby grants to You a perpetual,
75
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
76
+ (except as stated in this section) patent license to make, have made,
77
+ use, offer to sell, sell, import, and otherwise transfer the Work,
78
+ where such license applies only to those patent claims licensable
79
+ by such Contributor that are necessarily infringed by their
80
+ Contribution(s) alone or by combination of their Contribution(s)
81
+ with the Work to which such Contribution(s) was submitted. If You
82
+ institute patent litigation against any entity (including a
83
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
84
+ or a Contribution incorporated within the Work constitutes direct
85
+ or contributory patent infringement, then any patent licenses
86
+ granted to You under this License for that Work shall terminate
87
+ as of the date such litigation is filed.
88
+
89
+ 4. Redistribution. You may reproduce and distribute copies of the
90
+ Work or Derivative Works thereof in any medium, with or without
91
+ modifications, and in Source or Object form, provided that You
92
+ meet the following conditions:
93
+
94
+ (a) You must give any other recipients of the Work or
95
+ Derivative Works a copy of this License; and
96
+
97
+ (b) You must cause any modified files to carry prominent notices
98
+ stating that You changed the files; and
99
+
100
+ (c) You must retain, in the Source form of any Derivative Works
101
+ that You distribute, all copyright, patent, trademark, and
102
+ attribution notices from the Source form of the Work,
103
+ excluding those notices that do not pertain to any part of
104
+ the Derivative Works; and
105
+
106
+ (d) If the Work includes a "NOTICE" text file as part of its
107
+ distribution, then any Derivative Works that You distribute must
108
+ include a readable copy of the attribution notices contained
109
+ within such NOTICE file, excluding those notices that do not
110
+ pertain to any part of the Derivative Works, in at least one
111
+ of the following places: within a NOTICE text file distributed
112
+ as part of the Derivative Works; within the Source form or
113
+ documentation, if provided along with the Derivative Works; or,
114
+ within a display generated by the Derivative Works, if and
115
+ wherever such third-party notices normally appear. The contents
116
+ of the NOTICE file are for informational purposes only and
117
+ do not modify the License. You may add Your own attribution
118
+ notices within Derivative Works that You distribute, alongside
119
+ or as an addendum to the NOTICE text from the Work, provided
120
+ that such additional attribution notices cannot be construed
121
+ as modifying the License.
122
+
123
+ You may add Your own copyright statement to Your modifications and
124
+ may provide additional or different license terms and conditions
125
+ for use, reproduction, or distribution of Your modifications, or
126
+ for any such Derivative Works as a whole, provided Your use,
127
+ reproduction, and distribution of the Work otherwise complies with
128
+ the conditions stated in this License.
129
+
130
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
131
+ any Contribution intentionally submitted for inclusion in the Work
132
+ by You to the Licensor shall be under the terms and conditions of
133
+ this License, without any additional terms or conditions.
134
+ Notwithstanding the above, nothing herein shall supersede or modify
135
+ the terms of any separate license agreement you may have executed
136
+ with Licensor regarding such Contributions.
137
+
138
+ 6. Trademarks. This License does not grant permission to use the trade
139
+ names, trademarks, service marks, or product names of the Licensor,
140
+ except as required for reasonable and customary use in describing the
141
+ origin of the Work and reproducing the content of the NOTICE file.
142
+
143
+ 7. Disclaimer of Warranty. Unless required by applicable law or
144
+ agreed to in writing, Licensor provides the Work (and each
145
+ Contributor provides its Contributions) on an "AS IS" BASIS,
146
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
147
+ implied, including, without limitation, any warranties or conditions
148
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
149
+ PARTICULAR PURPOSE. You are solely responsible for determining the
150
+ appropriateness of using or redistributing the Work and assume any
151
+ risks associated with Your exercise of permissions under this License.
152
+
153
+ 8. Limitation of Liability. In no event and under no legal theory,
154
+ whether in tort (including negligence), contract, or otherwise,
155
+ unless required by applicable law (such as deliberate and grossly
156
+ negligent acts) or agreed to in writing, shall any Contributor be
157
+ liable to You for damages, including any direct, indirect, special,
158
+ incidental, or consequential damages of any character arising as a
159
+ result of this License or out of the use or inability to use the
160
+ Work (including but not limited to damages for loss of goodwill,
161
+ work stoppage, computer failure or malfunction, or any and all
162
+ other commercial damages or losses), even if such Contributor
163
+ has been advised of the possibility of such damages.
164
+
165
+ 9. Accepting Warranty or Additional Liability. While redistributing
166
+ the Work or Derivative Works thereof, You may choose to offer,
167
+ and charge a fee for, acceptance of support, warranty, indemnity,
168
+ or other liability obligations and/or rights consistent with this
169
+ License. However, in accepting such obligations, You may act only
170
+ on Your own behalf and on Your sole responsibility, not on behalf
171
+ of any other Contributor, and only if You agree to indemnify,
172
+ defend, and hold each Contributor harmless for any liability
173
+ incurred by, or claims asserted against, such Contributor by reason
174
+ of your accepting any such warranty or additional liability.
175
+
176
+ END OF TERMS AND CONDITIONS
177
+
178
+ APPENDIX: How to apply the Apache License to your work.
179
+
180
+ To apply the Apache License to your work, attach the following
181
+ boilerplate notice, with the fields enclosed by brackets "[]"
182
+ replaced with your own identifying information. (Don't include
183
+ the brackets!) The text should be enclosed in the appropriate
184
+ comment syntax for the file format. We also recommend that a
185
+ file or class name and description of purpose be included on the
186
+ same "printed page" as the copyright notice for easier
187
+ identification within third-party archives.
188
+
189
+ Copyright [yyyy] [name of copyright owner]
190
+
191
+ Licensed under the Apache License, Version 2.0 (the "License");
192
+ you may not use this file except in compliance with the License.
193
+ You may obtain a copy of the License at
194
+
195
+ http://www.apache.org/licenses/LICENSE-2.0
196
+
197
+ Unless required by applicable law or agreed to in writing, software
198
+ distributed under the License is distributed on an "AS IS" BASIS,
199
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
200
+ See the License for the specific language governing permissions and
201
+ limitations under the License.
Makefile ADDED
@@ -0,0 +1,132 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ MODULE := github.com/metatube-community/metatube-sdk-go
2
+
3
+ SERVER_NAME := metatube-server
4
+ SERVER_CODE := cmd/server/main.go
5
+
6
+ BUILD_DIR := build
7
+ BUILD_TAGS :=
8
+ BUILD_FLAGS := -v
9
+ BUILD_COMMIT := $(shell git rev-parse --short HEAD)
10
+ BUILD_VERSION := $(shell git describe --abbrev=0 --tags HEAD | cut -d'v' -f 2)
11
+
12
+ CGO_ENABLED := 0
13
+ GO111MODULE := on
14
+
15
+ LDFLAGS += -w -s -buildid=
16
+ LDFLAGS += -X "$(MODULE)/internal/version.Version=$(BUILD_VERSION)"
17
+ LDFLAGS += -X "$(MODULE)/internal/version.GitCommit=$(BUILD_COMMIT)"
18
+
19
+ GO_BUILD = GO111MODULE=$(GO111MODULE) CGO_ENABLED=$(CGO_ENABLED) \
20
+ go build $(BUILD_FLAGS) -ldflags '$(LDFLAGS)' -tags '$(BUILD_TAGS)' -trimpath
21
+
22
+ UNIX_ARCH_LIST = \
23
+ darwin-amd64 \
24
+ darwin-amd64-v3 \
25
+ darwin-arm64 \
26
+ freebsd-amd64 \
27
+ freebsd-amd64-v3 \
28
+ freebsd-arm64 \
29
+ linux-386 \
30
+ linux-amd64 \
31
+ linux-amd64-v3 \
32
+ linux-arm64 \
33
+ linux-armv5 \
34
+ linux-armv6 \
35
+ linux-armv7 \
36
+ linux-ppc64le \
37
+ linux-s390x \
38
+ openbsd-amd64 \
39
+ openbsd-amd64-v3
40
+
41
+ WINDOWS_ARCH_LIST = \
42
+ windows-amd64 \
43
+ windows-amd64-v3 \
44
+ windows-arm64
45
+
46
+ all: development
47
+
48
+ development: BUILD_TAGS += experimental
49
+ development:
50
+ $(GO_BUILD) -o $(BUILD_DIR)/$(SERVER_NAME) $(SERVER_CODE)
51
+
52
+ server:
53
+ $(GO_BUILD) -o $(BUILD_DIR)/$(SERVER_NAME) $(SERVER_CODE)
54
+
55
+ darwin-amd64:
56
+ GOARCH=amd64 GOOS=darwin $(GO_BUILD) -o $(BUILD_DIR)/$(SERVER_NAME)-$@ $(SERVER_CODE)
57
+
58
+ darwin-amd64-v3:
59
+ GOARCH=amd64 GOOS=darwin GOAMD64=v3 $(GO_BUILD) -o $(BUILD_DIR)/$(SERVER_NAME)-$@ $(SERVER_CODE)
60
+
61
+ darwin-arm64:
62
+ GOARCH=arm64 GOOS=darwin $(GO_BUILD) -o $(BUILD_DIR)/$(SERVER_NAME)-$@ $(SERVER_CODE)
63
+
64
+ freebsd-amd64:
65
+ GOARCH=amd64 GOOS=freebsd $(GO_BUILD) -o $(BUILD_DIR)/$(SERVER_NAME)-$@ $(SERVER_CODE)
66
+
67
+ freebsd-amd64-v3:
68
+ GOARCH=amd64 GOOS=freebsd GOAMD64=v3 $(GO_BUILD) -o $(BUILD_DIR)/$(SERVER_NAME)-$@ $(SERVER_CODE)
69
+
70
+ freebsd-arm64:
71
+ GOARCH=arm64 GOOS=freebsd $(GO_BUILD) -o $(BUILD_DIR)/$(SERVER_NAME)-$@ $(SERVER_CODE)
72
+
73
+ linux-386:
74
+ GOARCH=386 GOOS=linux $(GO_BUILD) -o $(BUILD_DIR)/$(SERVER_NAME)-$@ $(SERVER_CODE)
75
+
76
+ linux-amd64:
77
+ GOARCH=amd64 GOOS=linux $(GO_BUILD) -o $(BUILD_DIR)/$(SERVER_NAME)-$@ $(SERVER_CODE)
78
+
79
+ linux-amd64-v3:
80
+ GOARCH=amd64 GOOS=linux GOAMD64=v3 $(GO_BUILD) -o $(BUILD_DIR)/$(SERVER_NAME)-$@ $(SERVER_CODE)
81
+
82
+ linux-arm64:
83
+ GOARCH=arm64 GOOS=linux $(GO_BUILD) -o $(BUILD_DIR)/$(SERVER_NAME)-$@ $(SERVER_CODE)
84
+
85
+ linux-armv5:
86
+ GOARCH=arm GOARM=5 GOOS=linux $(GO_BUILD) -o $(BUILD_DIR)/$(SERVER_NAME)-$@ $(SERVER_CODE)
87
+
88
+ linux-armv6:
89
+ GOARCH=arm GOARM=6 GOOS=linux $(GO_BUILD) -o $(BUILD_DIR)/$(SERVER_NAME)-$@ $(SERVER_CODE)
90
+
91
+ linux-armv7:
92
+ GOARCH=arm GOARM=7 GOOS=linux $(GO_BUILD) -o $(BUILD_DIR)/$(SERVER_NAME)-$@ $(SERVER_CODE)
93
+
94
+ linux-ppc64le:
95
+ GOARCH=ppc64le GOOS=linux $(GO_BUILD) -o $(BUILD_DIR)/$(SERVER_NAME)-$@ $(SERVER_CODE)
96
+
97
+ linux-s390x:
98
+ GOARCH=s390x GOOS=linux $(GO_BUILD) -o $(BUILD_DIR)/$(SERVER_NAME)-$@ $(SERVER_CODE)
99
+
100
+ openbsd-amd64:
101
+ GOARCH=amd64 GOOS=openbsd $(GO_BUILD) -o $(BUILD_DIR)/$(SERVER_NAME)-$@ $(SERVER_CODE)
102
+
103
+ openbsd-amd64-v3:
104
+ GOARCH=amd64 GOOS=openbsd GOAMD64=v3 $(GO_BUILD) -o $(BUILD_DIR)/$(SERVER_NAME)-$@ $(SERVER_CODE)
105
+
106
+ windows-amd64:
107
+ GOARCH=amd64 GOOS=windows $(GO_BUILD) -o $(BUILD_DIR)/$(SERVER_NAME)-$@.exe $(SERVER_CODE)
108
+
109
+ windows-amd64-v3:
110
+ GOARCH=amd64 GOOS=windows GOAMD64=v3 $(GO_BUILD) -o $(BUILD_DIR)/$(SERVER_NAME)-$@.exe $(SERVER_CODE)
111
+
112
+ windows-arm64:
113
+ GOARCH=arm64 GOOS=windows $(GO_BUILD) -o $(BUILD_DIR)/$(SERVER_NAME)-$@.exe $(SERVER_CODE)
114
+
115
+ unix_releases := $(addsuffix .zip, $(UNIX_ARCH_LIST))
116
+ windows_releases := $(addsuffix .zip, $(WINDOWS_ARCH_LIST))
117
+
118
+ $(unix_releases): %.zip: %
119
+ @zip -qmj $(BUILD_DIR)/$(SERVER_NAME)-$(basename $@).zip $(BUILD_DIR)/$(SERVER_NAME)-$(basename $@)
120
+
121
+ $(windows_releases): %.zip: %
122
+ @zip -qmj $(BUILD_DIR)/$(SERVER_NAME)-$(basename $@).zip $(BUILD_DIR)/$(SERVER_NAME)-$(basename $@).exe
123
+
124
+ all-arch: $(UNIX_ARCH_LIST) $(WINDOWS_ARCH_LIST)
125
+
126
+ releases: $(unix_releases) $(windows_releases)
127
+
128
+ lint:
129
+ golangci-lint run ./...
130
+
131
+ clean:
132
+ rm -rf $(BUILD_DIR)
README.md ADDED
@@ -0,0 +1,73 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ title: MetaTube
3
+ emoji: 📽️
4
+ colorFrom: indigo
5
+ colorTo: blue
6
+ sdk: docker
7
+ pinned: false
8
+ ---
9
+
10
+ # MetaTube SDK Go
11
+
12
+ [![Build Status](https://img.shields.io/github/actions/workflow/status/metatube-community/metatube-sdk-go/docker.yml?branch=main&style=flat-square&logo=github-actions)](https://github.com/metatube-community/metatube-sdk-go/actions/workflows/release.yml)
13
+ [![Go Report Card](https://goreportcard.com/badge/github.com/metatube-community/metatube-sdk-go?style=flat-square)](https://github.com/metatube-community/metatube-sdk-go)
14
+ [![Require Go Version](https://img.shields.io/badge/go-%3E%3D1.25-30dff3?style=flat-square&logo=go)](https://github.com/metatube-community/metatube-sdk-go/blob/main/go.mod)
15
+ [![GitHub License](https://img.shields.io/github/license/metatube-community/metatube-sdk-go?color=e4682a&logo=apache&style=flat-square)](https://github.com/metatube-community/metatube-sdk-go/blob/main/LICENSE)
16
+ [![Tag](https://img.shields.io/github/v/tag/metatube-community/metatube-sdk-go?color=%23ff8936&logo=fitbit&style=flat-square)](https://github.com/metatube-community/metatube-sdk-go/tags)
17
+
18
+ Metadata Tube SDK in Golang.
19
+
20
+ ## Contents
21
+
22
+ - [MetaTube SDK Go](#metatube-sdk-go)
23
+ - [Contents](#contents)
24
+ - [Features](#features)
25
+ - [Installation](#installation)
26
+ - [Credits](#credits)
27
+ - [License](#license)
28
+
29
+ ## Features
30
+
31
+ - Supported platforms
32
+ - Linux
33
+ - Darwin
34
+ - Windows
35
+ - BSD(s)
36
+ - Supported Databases
37
+ - [SQLite](https://gitlab.com/cznic/sqlite)
38
+ - [PostgreSQL](https://github.com/jackc/pgx)
39
+ - Image processing
40
+ - Auto cropping
41
+ - Badge support
42
+ - Face detection
43
+ - Image hashing
44
+ - RESTful API
45
+ - 20+ providers
46
+ - Text translation
47
+
48
+ ## Installation
49
+
50
+ To install this package, you first need [Go](https://golang.org/) installed (**go1.25+ is required**), then you can use
51
+ the below Go command to install SDK.
52
+
53
+ ```sh
54
+ go get -u github.com/metatube-community/metatube-sdk-go
55
+ ```
56
+
57
+ ## Credits
58
+
59
+ | Library | Description |
60
+ |-----------------------------------------------------------------|------------------------------------------------------------------------------------------------------|
61
+ | [gocolly/colly](https://github.com/gocolly/colly) | Elegant Scraper and Crawler Framework for Golang |
62
+ | [gin-gonic/gin](https://github.com/gin-gonic/gin) | Gin is a HTTP web framework written in Go |
63
+ | [gorm.io/gorm](https://gorm.io/) | The fantastic ORM library for Golang |
64
+ | [esimov/pigo](https://github.com/esimov/pigo) | Fast face detection, pupil/eyes localization and facial landmark points detection library in pure Go |
65
+ | [robertkrimen/otto](https://github.com/robertkrimen/otto) | A JavaScript interpreter in Go (golang) |
66
+ | [modernc.org/sqlite](https://gitlab.com/cznic/sqlite) | Package sqlite is a CGo-free port of SQLite/SQLite3 |
67
+ | [corona10/goimagehash](https://github.com/corona10/goimagehash) | Go Perceptual image hashing package |
68
+ | [antchfx/xpath](https://github.com/antchfx/xpath) | XPath package for Golang, supports HTML, XML, JSON document query |
69
+ | [gen2brain/jpegli](https://github.com/gen2brain/jpegli) | Go encoder/decoder for JPEG based on jpegli |
70
+
71
+ ## License
72
+
73
+ [Apache-2.0 License](https://github.com/metatube-community/metatube-sdk-go/blob/main/LICENSE)
cmd/cmd.go ADDED
@@ -0,0 +1,111 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ package cmd
2
+
3
+ import (
4
+ goflag "flag"
5
+ "log"
6
+ "os"
7
+ "time"
8
+
9
+ "github.com/gin-gonic/gin"
10
+ "github.com/peterbourgon/ff/v3"
11
+
12
+ "github.com/metatube-community/metatube-sdk-go/database"
13
+ "github.com/metatube-community/metatube-sdk-go/engine"
14
+ "github.com/metatube-community/metatube-sdk-go/internal/envconfig"
15
+ "github.com/metatube-community/metatube-sdk-go/route"
16
+ "github.com/metatube-community/metatube-sdk-go/route/auth"
17
+ )
18
+
19
+ var Config = &struct {
20
+ // main config
21
+ Bind string
22
+ Port string
23
+ Token string
24
+ DSN string
25
+
26
+ // engine config
27
+ RequestTimeout time.Duration
28
+
29
+ // database config
30
+ DBMaxIdleConns int
31
+ DBMaxOpenConns int
32
+ DBAutoMigrate bool
33
+ DBPreparedStmt bool
34
+
35
+ // version flag
36
+ VersionFlag bool
37
+ }{}
38
+
39
+ func init() {
40
+ // gin init
41
+ gin.DisableConsoleColor()
42
+
43
+ // flag init
44
+ flag := goflag.NewFlagSet("", goflag.ExitOnError)
45
+
46
+ // flag parse
47
+ flag.StringVar(&Config.Bind, "bind", "", "Bind address of server")
48
+ flag.StringVar(&Config.Port, "port", "8080", "Port number of server")
49
+ flag.StringVar(&Config.Token, "token", "", "Token to access server")
50
+ flag.StringVar(&Config.DSN, "dsn", "", "Database Service Name")
51
+ flag.DurationVar(&Config.RequestTimeout, "request-timeout", engine.DefaultRequestTimeout, "Timeout per request")
52
+ flag.IntVar(&Config.DBMaxIdleConns, "db-max-idle-conns", 0, "Database max idle connections")
53
+ flag.IntVar(&Config.DBMaxOpenConns, "db-max-open-conns", 0, "Database max open connections")
54
+ flag.BoolVar(&Config.DBAutoMigrate, "db-auto-migrate", false, "Database auto migration")
55
+ flag.BoolVar(&Config.DBPreparedStmt, "db-prepared-stmt", false, "Database prepared statement")
56
+ flag.BoolVar(&Config.VersionFlag, "version", false, "Show version")
57
+ ff.Parse(flag, os.Args[1:], ff.WithEnvVars())
58
+ }
59
+
60
+ func Router(names ...string) *gin.Engine {
61
+ db, err := database.Open(&database.Config{
62
+ DSN: Config.DSN,
63
+ PreparedStmt: Config.DBPreparedStmt,
64
+ MaxIdleConns: Config.DBMaxIdleConns,
65
+ MaxOpenConns: Config.DBMaxOpenConns,
66
+ DisableAutomaticPing: true,
67
+ })
68
+ if err != nil {
69
+ log.Fatal(err)
70
+ }
71
+
72
+ // engine options
73
+ var opts []engine.Option
74
+
75
+ // timeout must >= 1 second
76
+ if Config.RequestTimeout >= time.Second {
77
+ opts = append(opts, engine.WithRequestTimeout(Config.RequestTimeout))
78
+ }
79
+
80
+ // specify engine name
81
+ for _, name := range names {
82
+ opts = append(opts, engine.WithEngineName(name))
83
+ }
84
+
85
+ // // set actor provider configs if any
86
+ for provider, config := range envconfig.ActorProviderConfigs.Iterator() {
87
+ opts = append(opts, engine.WithActorProviderConfig(provider, config))
88
+ }
89
+
90
+ // set movie provider configs if any
91
+ for provider, config := range envconfig.MovieProviderConfigs.Iterator() {
92
+ opts = append(opts, engine.WithMovieProviderConfig(provider, config))
93
+ }
94
+
95
+ app := engine.New(db, opts...)
96
+
97
+ // always enable auto migrate for sqlite DB
98
+ if app.DBDriver() == database.Sqlite {
99
+ Config.DBAutoMigrate = true
100
+ }
101
+ if err = app.DBAutoMigrate(Config.DBAutoMigrate); err != nil {
102
+ log.Fatal(err)
103
+ }
104
+
105
+ var token auth.Validator
106
+ if Config.Token != "" {
107
+ token = auth.Token(Config.Token)
108
+ }
109
+
110
+ return route.New(app, token)
111
+ }
cmd/init.go ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ package cmd
2
+
3
+ import (
4
+ "go.uber.org/automaxprocs/maxprocs"
5
+ )
6
+
7
+ func init() {
8
+ maxprocs.Set(maxprocs.Logger(func(string, ...any) {}))
9
+ }
cmd/server/main.go ADDED
@@ -0,0 +1,35 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ package main
2
+
3
+ import (
4
+ "fmt"
5
+ "log"
6
+ "net"
7
+ "net/http"
8
+ "os"
9
+
10
+ "github.com/metatube-community/metatube-sdk-go/cmd"
11
+ "github.com/metatube-community/metatube-sdk-go/engine"
12
+ V "github.com/metatube-community/metatube-sdk-go/internal/version"
13
+ )
14
+
15
+ func showVersionAndExit() {
16
+ fmt.Println(V.BuildString())
17
+ os.Exit(0)
18
+ }
19
+
20
+ func main() {
21
+ if _, isSet := os.LookupEnv("VERSION"); cmd.Config.VersionFlag &&
22
+ !isSet /* NOTE: ignore this flag if ENV contains VERSION variable. */ {
23
+ showVersionAndExit()
24
+ }
25
+
26
+ var (
27
+ addr = net.JoinHostPort(
28
+ cmd.Config.Bind,
29
+ cmd.Config.Port)
30
+ router = cmd.Router(engine.DefaultEngineName)
31
+ )
32
+ if err := http.ListenAndServe(addr, router); err != nil {
33
+ log.Fatal(err)
34
+ }
35
+ }
collection/maps/cimap.go ADDED
@@ -0,0 +1,96 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ package maps
2
+
3
+ import (
4
+ "encoding/json"
5
+ "iter"
6
+
7
+ "github.com/projectbarks/cimap"
8
+ )
9
+
10
+ var (
11
+ _ json.Marshaler = (*CaseInsensitiveMap[any])(nil)
12
+ _ json.Unmarshaler = (*CaseInsensitiveMap[any])(nil)
13
+ )
14
+
15
+ type CaseInsensitiveMap[T any] struct {
16
+ internalMap *cimap.CaseInsensitiveMap[T]
17
+ }
18
+
19
+ func NewCaseInsensitiveMap[T any]() *CaseInsensitiveMap[T] {
20
+ return &CaseInsensitiveMap[T]{
21
+ internalMap: cimap.New[T](),
22
+ }
23
+ }
24
+
25
+ func NewCaseInsensitiveMapWithCapacity[T any](capacity int) *CaseInsensitiveMap[T] {
26
+ return &CaseInsensitiveMap[T]{
27
+ internalMap: cimap.New[T](capacity),
28
+ }
29
+ }
30
+
31
+ func (m *CaseInsensitiveMap[T]) Copy() *CaseInsensitiveMap[T] {
32
+ m2 := NewCaseInsensitiveMapWithCapacity[T](m.Len())
33
+ for key, value := range m.Iterator() {
34
+ m2.Set(key, value)
35
+ }
36
+ return m2
37
+ }
38
+
39
+ func (m *CaseInsensitiveMap[T]) Has(key string) bool {
40
+ _, exist := m.internalMap.Get(key)
41
+ return exist
42
+ }
43
+
44
+ func (m *CaseInsensitiveMap[T]) Get(key string) (T, bool) {
45
+ return m.internalMap.Get(key)
46
+ }
47
+
48
+ func (m *CaseInsensitiveMap[T]) GetOrDefault(key string, defaultValues ...T) T {
49
+ value, exist := m.internalMap.Get(key)
50
+ if exist {
51
+ return value
52
+ }
53
+ if len(defaultValues) > 0 {
54
+ return defaultValues[0]
55
+ }
56
+ var defaultValue T
57
+ return defaultValue
58
+ }
59
+
60
+ func (m *CaseInsensitiveMap[T]) Set(key string, value T) {
61
+ m.internalMap.Add(key, value)
62
+ }
63
+
64
+ func (m *CaseInsensitiveMap[T]) Delete(key string) {
65
+ m.internalMap.Delete(key)
66
+ }
67
+
68
+ func (m *CaseInsensitiveMap[T]) Len() int {
69
+ return m.internalMap.Len()
70
+ }
71
+
72
+ func (m *CaseInsensitiveMap[T]) Keys() iter.Seq[string] {
73
+ return m.internalMap.Keys()
74
+ }
75
+
76
+ func (m *CaseInsensitiveMap[T]) Values() iter.Seq[T] {
77
+ return func(yield func(T) bool) {
78
+ for _, value := range m.Iterator() {
79
+ if !yield(value) {
80
+ return
81
+ }
82
+ }
83
+ }
84
+ }
85
+
86
+ func (m *CaseInsensitiveMap[T]) Iterator() iter.Seq2[string, T] {
87
+ return m.internalMap.Iterator()
88
+ }
89
+
90
+ func (m *CaseInsensitiveMap[T]) MarshalJSON() ([]byte, error) {
91
+ return m.internalMap.MarshalJSON()
92
+ }
93
+
94
+ func (m *CaseInsensitiveMap[T]) UnmarshalJSON(data []byte) error {
95
+ return m.internalMap.UnmarshalJSON(data)
96
+ }
collection/maps/cimap_test.go ADDED
@@ -0,0 +1,90 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ package maps
2
+
3
+ import (
4
+ "encoding/json"
5
+ "slices"
6
+ "testing"
7
+
8
+ "github.com/stretchr/testify/assert"
9
+ )
10
+
11
+ func TestCaseInsensitiveMap(t *testing.T) {
12
+ m := NewCaseInsensitiveMap[string]()
13
+
14
+ // Set with mixed casing
15
+ m.Set("FOO", "bar")
16
+ m.Set("Baz", "qux")
17
+ assert.Equal(t, 2, m.Len())
18
+
19
+ // Should be case-insensitive
20
+ val, ok := m.Get("foo")
21
+ assert.True(t, ok)
22
+ assert.Equal(t, "bar", val)
23
+
24
+ val, ok = m.Get("baz")
25
+ assert.True(t, ok)
26
+ assert.Equal(t, "qux", val)
27
+
28
+ val, ok = m.Get("BAZ")
29
+ assert.True(t, ok)
30
+ assert.Equal(t, "qux", val)
31
+
32
+ exist := m.Has("foo")
33
+ assert.True(t, exist)
34
+
35
+ exist = m.Has("baz")
36
+ assert.True(t, exist)
37
+
38
+ exist = m.Has("BAZ")
39
+ assert.True(t, exist)
40
+
41
+ val = m.GetOrDefault("foo", "quux")
42
+ assert.Equal(t, "bar", val)
43
+
44
+ val = m.GetOrDefault("baz", "quux")
45
+ assert.Equal(t, "qux", val)
46
+
47
+ val = m.GetOrDefault("bar", "quux")
48
+ assert.Equal(t, "quux", val)
49
+
50
+ val = m.GetOrDefault("bar")
51
+ assert.Equal(t, "", val)
52
+
53
+ keys := slices.Collect(m.Keys())
54
+ slices.Sort(keys)
55
+ assert.Equal(t, []string{"Baz", "FOO"}, keys)
56
+
57
+ values := slices.Collect(m.Values())
58
+ slices.Sort(values)
59
+ assert.Equal(t, []string{"bar", "qux"}, values)
60
+
61
+ // Delete should also be case-insensitive
62
+ m.Delete("FOO")
63
+ _, ok = m.Get("foo")
64
+ assert.False(t, ok)
65
+ assert.Equal(t, 1, m.Len())
66
+
67
+ // Test JSON marshal/unmarshal
68
+ data, err := json.Marshal(m)
69
+ if assert.NoError(t, err) {
70
+ assert.JSONEq(t, `{
71
+ "Baz":"qux"
72
+ }`, string(data))
73
+ }
74
+
75
+ copied := m.Copy()
76
+ data2, err := json.Marshal(copied)
77
+ if assert.NoError(t, err) {
78
+ assert.JSONEq(t, `{
79
+ "Baz":"qux"
80
+ }`, string(data2))
81
+ }
82
+
83
+ m2 := NewCaseInsensitiveMapWithCapacity[string](m.Len())
84
+ err = json.Unmarshal(data, m2)
85
+ if assert.NoError(t, err) {
86
+ val, ok = m2.Get("baz")
87
+ assert.True(t, ok)
88
+ assert.Equal(t, "qux", val)
89
+ }
90
+ }
collection/maps/orderedmap.go ADDED
@@ -0,0 +1,79 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ package maps
2
+
3
+ import (
4
+ "bytes"
5
+ "encoding/json"
6
+ "iter"
7
+
8
+ "github.com/elliotchance/orderedmap/v3"
9
+ jsoniter "github.com/json-iterator/go"
10
+ )
11
+
12
+ var (
13
+ _ json.Marshaler = (*OrderedMap[int, any])(nil)
14
+ _ json.Unmarshaler = (*OrderedMap[int, any])(nil)
15
+ )
16
+
17
+ type OrderedMap[K comparable, V any] struct {
18
+ *orderedmap.OrderedMap[K, V]
19
+ escapeHTML bool
20
+ }
21
+
22
+ func NewOrderedMap[K comparable, V any]() *OrderedMap[K, V] {
23
+ return &OrderedMap[K, V]{OrderedMap: orderedmap.NewOrderedMap[K, V]()}
24
+ }
25
+
26
+ func (m *OrderedMap[K, V]) SetEscapeHTML(on bool) {
27
+ m.escapeHTML = on
28
+ }
29
+
30
+ func (m *OrderedMap[K, V]) Copy() *OrderedMap[K, V] {
31
+ return &OrderedMap[K, V]{
32
+ OrderedMap: m.OrderedMap.Copy(),
33
+ escapeHTML: m.escapeHTML,
34
+ }
35
+ }
36
+
37
+ func (m *OrderedMap[K, V]) Iterator() iter.Seq2[K, V] {
38
+ return m.AllFromFront() // For compatibility.
39
+ }
40
+
41
+ func (m *OrderedMap[K, V]) MarshalJSON() ([]byte, error) {
42
+ buf := bytes.NewBuffer(nil)
43
+ buf.WriteByte('{')
44
+ enc := json.NewEncoder(buf)
45
+ enc.SetEscapeHTML(m.escapeHTML)
46
+ for el := m.Front(); el != nil; el = el.Next() {
47
+ if el != m.Front() {
48
+ buf.WriteByte(',')
49
+ }
50
+ // add key
51
+ if err := enc.Encode(el.Key); err != nil {
52
+ return nil, err
53
+ }
54
+ buf.WriteByte(':')
55
+ // add value
56
+ if err := enc.Encode(el.Value); err != nil {
57
+ return nil, err
58
+ }
59
+ }
60
+ buf.WriteByte('}')
61
+ return buf.Bytes(), nil
62
+ }
63
+
64
+ func (m *OrderedMap[K, V]) UnmarshalJSON(data []byte) error {
65
+ if m.OrderedMap == nil {
66
+ m.OrderedMap = orderedmap.NewOrderedMap[K, V]()
67
+ }
68
+ temp := make(map[K]V)
69
+ defer clear(temp) // for gc
70
+ if err := json.Unmarshal(data, &temp); err != nil {
71
+ return err
72
+ }
73
+ root := jsoniter.Get(data)
74
+ for _, key := range root.Keys() {
75
+ k := any(key).(K)
76
+ m.Set(k, temp[k])
77
+ }
78
+ return nil
79
+ }
collection/maps/orderedmap_test.go ADDED
@@ -0,0 +1,112 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ package maps
2
+
3
+ import (
4
+ "encoding/json"
5
+ "slices"
6
+ "testing"
7
+
8
+ "github.com/stretchr/testify/assert"
9
+ )
10
+
11
+ func TestOrderedMap(t *testing.T) {
12
+ t.Run("JSON Marshal", func(t *testing.T) {
13
+ m := NewOrderedMap[string, any]()
14
+ b, _ := json.Marshal(m)
15
+ assert.JSONEq(t, `{}`, string(b))
16
+
17
+ m.Set("a", 1)
18
+ m.Set("c", "2")
19
+ m.Set("b", 3.0)
20
+ m.Set("b", 1.5)
21
+ assert.Equal(t, []any{1, "2", 1.5}, slices.Collect(m.Values()))
22
+
23
+ b, _ = json.Marshal(m)
24
+ assert.JSONEq(t, `{
25
+ "a":1,
26
+ "c":"2",
27
+ "b":1.5
28
+ }`, string(b))
29
+ })
30
+
31
+ t.Run("fixed type map unmarshal", func(t *testing.T) {
32
+ jsonData := `{
33
+ "a":1,
34
+ "c":2,
35
+ "b":0
36
+ }`
37
+ m := NewOrderedMap[string, int]()
38
+ err := m.UnmarshalJSON([]byte(jsonData))
39
+ if assert.NoError(t, err) {
40
+ b, _ := json.Marshal(m)
41
+ assert.JSONEq(t, `{"a":1,"c":2,"b":0}`, string(b))
42
+ }
43
+ })
44
+
45
+ t.Run("any type map unmarshal", func(t *testing.T) {
46
+ jsonData := `{
47
+ "a":1,
48
+ "c":"2",
49
+ "b":1.5,
50
+ "?":{"x":"y","j":"k","3":2}
51
+ }`
52
+ m := NewOrderedMap[string, any]()
53
+ err := m.UnmarshalJSON([]byte(jsonData))
54
+ if assert.NoError(t, err) {
55
+ b, _ := json.Marshal(m)
56
+ assert.JSONEq(t, `{
57
+ "a":1,"c":"2","b":1.5,
58
+ "?":{"3":2,"j":"k","x":"y"}
59
+ }`, string(b))
60
+ }
61
+ })
62
+
63
+ t.Run("Sorted sub map unmarshal", func(t *testing.T) {
64
+ jsonData := `{
65
+ "w":{"n":3,"m":5},
66
+ "b":{"f":1,"j":0}
67
+ }`
68
+ m := NewOrderedMap[string, map[string]int]()
69
+ err := m.UnmarshalJSON([]byte(jsonData))
70
+ if assert.NoError(t, err) {
71
+ b, _ := json.Marshal(m)
72
+ assert.JSONEq(t, `{
73
+ "w":{"m":5,"n":3},
74
+ "b":{"f":1,"j":0}
75
+ }`, string(b))
76
+ }
77
+ })
78
+
79
+ t.Run("Ordered sub map unmarshal", func(t *testing.T) {
80
+ jsonData := `{
81
+ "w":{"n":3,"m":5},
82
+ "b":{"f":1,"j":0}
83
+ }`
84
+ m := NewOrderedMap[string, *OrderedMap[string, int]]()
85
+ err := m.UnmarshalJSON([]byte(jsonData))
86
+ if assert.NoError(t, err) {
87
+ b, _ := json.Marshal(m)
88
+ assert.JSONEq(t, `{
89
+ "w":{"n":3,"m":5},
90
+ "b":{"f":1,"j":0}
91
+ }`, string(b))
92
+ }
93
+ })
94
+
95
+ t.Run("A lot of ordered sub maps unmarshal", func(t *testing.T) {
96
+ jsonData := `{
97
+ "w":{"n":{"g":3,"5":5},"m":{"v":3,"2":5}},
98
+ "b":{"f":{"h":3,"3":5},"j":{"x":3,"c":5}}
99
+ }`
100
+ m := NewOrderedMap[string, *OrderedMap[string, *OrderedMap[string, any]]]()
101
+ err := m.UnmarshalJSON([]byte(jsonData))
102
+ if assert.NoError(t, err) {
103
+ b, _ := json.Marshal(m)
104
+ assert.JSONEq(t, `{
105
+ "w":{"n":{"g":3,"5":5},
106
+ "m":{"v":3,"2":5}},
107
+ "b":{"f":{"h":3,"3":5},
108
+ "j":{"x":3,"c":5}}
109
+ }`, string(b))
110
+ }
111
+ })
112
+ }
collection/sets/orderedset.go ADDED
@@ -0,0 +1,76 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ package sets
2
+
3
+ import (
4
+ "encoding/json"
5
+ "iter"
6
+ "slices"
7
+
8
+ "github.com/elliotchance/orderedmap/v3"
9
+ )
10
+
11
+ var (
12
+ _ json.Marshaler = (*OrderedSet[int, any])(nil)
13
+ _ json.Unmarshaler = (*OrderedSet[int, any])(nil)
14
+ )
15
+
16
+ type OrderedSet[K comparable, V any] struct {
17
+ h func(V) K
18
+ m *orderedmap.OrderedMap[K, V]
19
+ }
20
+
21
+ func NewOrderedSet[T comparable]() *OrderedSet[T, T] {
22
+ return &OrderedSet[T, T]{
23
+ h: func(t T) T { return t },
24
+ m: orderedmap.NewOrderedMap[T, T](),
25
+ }
26
+ }
27
+
28
+ func NewOrderedSetWithHash[K comparable, V any](hash func(V) K) *OrderedSet[K, V] {
29
+ return &OrderedSet[K, V]{
30
+ h: hash,
31
+ m: orderedmap.NewOrderedMap[K, V](),
32
+ }
33
+ }
34
+
35
+ func (s *OrderedSet[K, V]) Len() int {
36
+ return s.m.Len()
37
+ }
38
+
39
+ func (s *OrderedSet[K, V]) Add(items ...V) {
40
+ for _, result := range items {
41
+ s.m.Set(s.h(result), result)
42
+ }
43
+ }
44
+
45
+ func (s *OrderedSet[K, V]) Del(items ...V) {
46
+ for _, result := range items {
47
+ s.m.Delete(s.h(result))
48
+ }
49
+ }
50
+
51
+ func (s *OrderedSet[K, V]) Iterator() iter.Seq[V] {
52
+ return func(yield func(V) bool) {
53
+ for _, v := range s.m.AllFromFront() {
54
+ if !yield(v) {
55
+ return
56
+ }
57
+ }
58
+ }
59
+ }
60
+
61
+ func (s *OrderedSet[K, V]) AsSlice() []V {
62
+ return slices.Collect(s.Iterator())
63
+ }
64
+
65
+ func (s *OrderedSet[K, V]) MarshalJSON() ([]byte, error) {
66
+ return json.Marshal(s.AsSlice())
67
+ }
68
+
69
+ func (s *OrderedSet[K, V]) UnmarshalJSON(data []byte) error {
70
+ vs := make([]V, 0)
71
+ if err := json.Unmarshal(data, &vs); err != nil {
72
+ return err
73
+ }
74
+ s.Add(vs...)
75
+ return nil
76
+ }
collection/sets/orderedset_test.go ADDED
@@ -0,0 +1,32 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ package sets
2
+
3
+ import (
4
+ "encoding/json"
5
+ "strconv"
6
+ "testing"
7
+
8
+ "github.com/stretchr/testify/assert"
9
+ )
10
+
11
+ func TestOrderedSet(t *testing.T) {
12
+ set := NewOrderedSet[int]()
13
+
14
+ set.Add(1, 4, 6, 8, 9)
15
+ set.Add(7, 4, 9, 2, 3)
16
+ assert.Equal(t, 8, set.Len())
17
+ assert.Equal(t, []int{1, 4, 6, 8, 9, 7, 2, 3}, set.AsSlice())
18
+
19
+ set.Del(4, 5, 6, 7)
20
+ assert.Equal(t, 5, set.Len())
21
+ assert.Equal(t, []int{1, 8, 9, 2, 3}, set.AsSlice())
22
+
23
+ b, _ := json.Marshal(set)
24
+ assert.JSONEq(t, `[1,8,9,2,3]`, string(b))
25
+
26
+ set2 := NewOrderedSetWithHash(func(v int) string {
27
+ return strconv.Itoa(v)
28
+ })
29
+ _ = json.Unmarshal(b, set2)
30
+ assert.Equal(t, 5, set.Len())
31
+ assert.Equal(t, []int{1, 8, 9, 2, 3}, set.AsSlice())
32
+ }
collection/slices/flatten.go ADDED
@@ -0,0 +1,18 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ package slices
2
+
3
+ import (
4
+ "slices"
5
+ )
6
+
7
+ // Flatten flattens a 2D slice into a 1D slice by merging all inner slices.
8
+ func Flatten[E any](s [][]E) []E {
9
+ return slices.Collect(func(yield func(E) bool) {
10
+ for _, i := range s {
11
+ for _, j := range i {
12
+ if !yield(j) {
13
+ return
14
+ }
15
+ }
16
+ }
17
+ })
18
+ }
collection/slices/flatten_test.go ADDED
@@ -0,0 +1,16 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ package slices
2
+
3
+ import (
4
+ "testing"
5
+
6
+ "github.com/stretchr/testify/assert"
7
+ )
8
+
9
+ func TestFlatten(t *testing.T) {
10
+ k := [][]int{{1, 2}, {3, 4}}
11
+ assert.Equal(t, []int{1, 2, 3, 4}, Flatten(k))
12
+
13
+ s := [][][]string{{{"a", "b"}, {"c", "d"}}, {{"e", "f"}, {"g", "h"}}}
14
+ assert.Equal(t, [][]string{{"a", "b"}, {"c", "d"}, {"e", "f"}, {"g", "h"}}, Flatten(s))
15
+ assert.Equal(t, []string{"a", "b", "c", "d", "e", "f", "g", "h"}, Flatten(Flatten(s)))
16
+ }
collection/slices/wslice.go ADDED
@@ -0,0 +1,48 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ package slices
2
+
3
+ import (
4
+ "cmp"
5
+ "sort"
6
+ )
7
+
8
+ var _ sort.Interface = (*WeightedSlice[any, int])(nil)
9
+
10
+ type WeightedSlice[O any, W cmp.Ordered] struct {
11
+ objects []O
12
+ weights []W
13
+ }
14
+
15
+ func NewWeightedSlice[O any, W cmp.Ordered](objects []O, weights []W) *WeightedSlice[O, W] {
16
+ if len(objects) != len(weights) {
17
+ panic("objects and weights must have the same length")
18
+ }
19
+ return &WeightedSlice[O, W]{objects, weights}
20
+ }
21
+
22
+ func (s *WeightedSlice[O, W]) Len() int {
23
+ return len(s.objects)
24
+ }
25
+
26
+ func (s *WeightedSlice[O, W]) Less(i int, j int) bool {
27
+ // higher-weighted item comes first.
28
+ return s.weights[i] > s.weights[j]
29
+ }
30
+
31
+ func (s *WeightedSlice[O, W]) Swap(i int, j int) {
32
+ s.weights[i], s.weights[j] = s.weights[j], s.weights[i]
33
+ s.objects[i], s.objects[j] = s.objects[j], s.objects[i]
34
+ }
35
+
36
+ func (s *WeightedSlice[O, W]) Append(object O, weight W) {
37
+ s.weights = append(s.weights, weight)
38
+ s.objects = append(s.objects, object)
39
+ }
40
+
41
+ func (s *WeightedSlice[O, W]) Slice() []O {
42
+ return s.objects
43
+ }
44
+
45
+ func (s *WeightedSlice[O, W]) SortFunc(sortFn func(p sort.Interface)) *WeightedSlice[O, W] {
46
+ sortFn(s)
47
+ return s
48
+ }
collection/slices/wslice_test.go ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ package slices
2
+
3
+ import (
4
+ "sort"
5
+ "testing"
6
+
7
+ "github.com/stretchr/testify/assert"
8
+ )
9
+
10
+ func TestWeightedSlice(t *testing.T) {
11
+ s := NewWeightedSlice(
12
+ // initialized pairs.
13
+ []int{9}, []float64{3},
14
+ )
15
+ s.Append(5, 2)
16
+ s.Append(1, 3)
17
+ s.Append(9, 1)
18
+ s.Append(-1, 6)
19
+ s.Append(9, 6)
20
+ s.Append(8, 6)
21
+ s.Append(7, 6)
22
+ s.Append(0, 6)
23
+ s.Append(12, 0)
24
+
25
+ exp := []int{-1, 9, 8, 7, 0, 9, 1, 5, 9, 12}
26
+ got := s.SortFunc(sort.Stable).Slice()
27
+ assert.Equal(t, exp, got)
28
+ }
collection/unionfind/README.md ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ # Union-Find
2
+
3
+ Package copied from: <https://github.com/moorara/algo/blob/main/unionfind/unionfind.go>
collection/unionfind/unionfind.go ADDED
@@ -0,0 +1,235 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ // Package unionfind implements union-find data structures and algorithms.
2
+ // It supports union and find queries.
3
+ //
4
+ // The union-find (a.k.a. disjoint-sets) data type is collection of n elements.
5
+ // Initially, each element belongs to exactly one set (n sets initially).
6
+ // Each set is represented by one element (canonical element, root, identifier, leader, or set representative).
7
+ // The union operation merges the set containing the element p with the set containing the element q.
8
+ // The find operation returns the canonical element of the set containing the element p.
9
+ //
10
+ // Elements in one set are considered connected to each other.
11
+ // "p is connected to q" is an equivalence relation:
12
+ //
13
+ // Reflexive: p is connected to p.
14
+ // Symmetric: If p is connected to q, then q is connected to p.
15
+ // Transitive: If p is connected to q and q is connected to r, then p is connected to r.
16
+ //
17
+ // An equivalence relation partitions the objects into equivalence classes.
18
+ package unionfind
19
+
20
+ // UnionFind is the interface a union-find data type.
21
+ type UnionFind interface {
22
+ Union(int, int)
23
+ Find(int) (int, bool)
24
+ IsConnected(int, int) bool
25
+ Count() int
26
+ }
27
+
28
+ type quickFind struct {
29
+ count int // number of components (equivalence classes)
30
+ id []int // determines component IDs (class representatives)
31
+ }
32
+
33
+ // NewQuickFind creates a new union-find data structure with quick find.
34
+ func NewQuickFind(n int) UnionFind {
35
+ id := make([]int, n)
36
+ for i := 0; i < n; i++ {
37
+ id[i] = i
38
+ }
39
+
40
+ return &quickFind{
41
+ count: n,
42
+ id: id,
43
+ }
44
+ }
45
+
46
+ func (u *quickFind) isValid(i int) bool {
47
+ return 0 <= i && i < len(u.id)
48
+ }
49
+
50
+ func (u *quickFind) Union(p, q int) {
51
+ if !u.isValid(p) || !u.isValid(q) {
52
+ return
53
+ }
54
+
55
+ pid, _ := u.Find(p)
56
+ qid, _ := u.Find(q)
57
+
58
+ if pid == qid {
59
+ return
60
+ }
61
+
62
+ // Rename p's component to q's id
63
+ for i := range u.id {
64
+ if u.id[i] == pid {
65
+ u.id[i] = qid
66
+ }
67
+ }
68
+
69
+ u.count--
70
+ }
71
+
72
+ func (u *quickFind) Find(p int) (int, bool) {
73
+ if !u.isValid(p) {
74
+ return -1, false
75
+ }
76
+
77
+ return u.id[p], true
78
+ }
79
+
80
+ func (u *quickFind) IsConnected(p, q int) bool {
81
+ if !u.isValid(p) || !u.isValid(q) {
82
+ return false
83
+ }
84
+
85
+ pid, _ := u.Find(p)
86
+ qid, _ := u.Find(q)
87
+
88
+ return pid == qid
89
+ }
90
+
91
+ func (u *quickFind) Count() int {
92
+ return u.count
93
+ }
94
+
95
+ type quickUnion struct {
96
+ count int // number of components (equivalence classes)
97
+ root []int // determines component parents (class representatives)
98
+ }
99
+
100
+ // NewQuickUnion creates a new union-find data structure with quick union.
101
+ func NewQuickUnion(n int) UnionFind {
102
+ root := make([]int, n)
103
+ for i := 0; i < n; i++ {
104
+ root[i] = i
105
+ }
106
+
107
+ return &quickUnion{
108
+ count: n,
109
+ root: root,
110
+ }
111
+ }
112
+
113
+ func (u *quickUnion) isValid(i int) bool {
114
+ return 0 <= i && i < len(u.root)
115
+ }
116
+
117
+ func (u *quickUnion) Union(p, q int) {
118
+ if !u.isValid(p) || !u.isValid(q) {
119
+ return
120
+ }
121
+
122
+ proot, _ := u.Find(p)
123
+ qroot, _ := u.Find(q)
124
+
125
+ if proot == qroot {
126
+ return
127
+ }
128
+
129
+ u.root[proot] = qroot
130
+ u.count--
131
+ }
132
+
133
+ func (u *quickUnion) Find(p int) (int, bool) {
134
+ if !u.isValid(p) {
135
+ return -1, false
136
+ }
137
+
138
+ for p != u.root[p] {
139
+ p = u.root[p]
140
+ }
141
+
142
+ return p, true
143
+ }
144
+
145
+ func (u *quickUnion) IsConnected(p, q int) bool {
146
+ if !u.isValid(p) || !u.isValid(q) {
147
+ return false
148
+ }
149
+
150
+ proot, _ := u.Find(p)
151
+ qroot, _ := u.Find(q)
152
+
153
+ return proot == qroot
154
+ }
155
+
156
+ func (u *quickUnion) Count() int {
157
+ return u.count
158
+ }
159
+
160
+ type weightedQuickUnion struct {
161
+ count int // number of components (equivalence classes)
162
+ root []int // determines component parents (class representatives)
163
+ size []int // number of elements in component (class) rooted at i
164
+ }
165
+
166
+ // NewWeightedQuickUnion creates a new weighted union-find data structure with quick union.
167
+ func NewWeightedQuickUnion(n int) UnionFind {
168
+ root := make([]int, n)
169
+ size := make([]int, n)
170
+ for i := 0; i < n; i++ {
171
+ root[i] = i
172
+ size[i] = 1
173
+ }
174
+
175
+ return &weightedQuickUnion{
176
+ count: n,
177
+ root: root,
178
+ size: size,
179
+ }
180
+ }
181
+
182
+ func (u *weightedQuickUnion) isValid(i int) bool {
183
+ return 0 <= i && i < len(u.root)
184
+ }
185
+
186
+ func (u *weightedQuickUnion) Union(p, q int) {
187
+ if !u.isValid(p) || !u.isValid(q) {
188
+ return
189
+ }
190
+
191
+ proot, _ := u.Find(p)
192
+ qroot, _ := u.Find(q)
193
+
194
+ if proot == qroot {
195
+ return
196
+ }
197
+
198
+ // make smaller root point to larger one
199
+ if u.size[proot] < u.size[qroot] {
200
+ u.root[proot] = qroot
201
+ u.size[qroot] += u.size[proot]
202
+ } else {
203
+ u.root[qroot] = proot
204
+ u.size[proot] += u.size[qroot]
205
+ }
206
+
207
+ u.count--
208
+ }
209
+
210
+ func (u *weightedQuickUnion) Find(p int) (int, bool) {
211
+ if !u.isValid(p) {
212
+ return -1, false
213
+ }
214
+
215
+ for p != u.root[p] {
216
+ p = u.root[p]
217
+ }
218
+
219
+ return p, true
220
+ }
221
+
222
+ func (u *weightedQuickUnion) IsConnected(p, q int) bool {
223
+ if !u.isValid(p) || !u.isValid(q) {
224
+ return false
225
+ }
226
+
227
+ proot, _ := u.Find(p)
228
+ qroot, _ := u.Find(q)
229
+
230
+ return proot == qroot
231
+ }
232
+
233
+ func (u *weightedQuickUnion) Count() int {
234
+ return u.count
235
+ }
collection/unionfind/unionfind_test.go ADDED
@@ -0,0 +1,374 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ package unionfind
2
+
3
+ import (
4
+ "testing"
5
+
6
+ "github.com/stretchr/testify/assert"
7
+ )
8
+
9
+ type (
10
+ findTest struct {
11
+ name string
12
+ p int
13
+ expectedID int
14
+ expectedOK bool
15
+ }
16
+
17
+ connectivityTest struct {
18
+ name string
19
+ p, q int
20
+ expectedIsConnected bool
21
+ }
22
+
23
+ unionFindTest struct {
24
+ name string
25
+ n int
26
+ unions [][2]int
27
+ expectedCount int
28
+ findTests []findTest
29
+ connectivityTests []connectivityTest
30
+ }
31
+ )
32
+
33
+ var runTest = func(t *testing.T, tc unionFindTest, uf UnionFind) {
34
+ for _, u := range tc.unions {
35
+ uf.Union(u[0], u[1])
36
+ }
37
+
38
+ t.Run("Count", func(t *testing.T) {
39
+ assert.Equal(t, tc.expectedCount, uf.Count())
40
+ })
41
+
42
+ t.Run("Find", func(t *testing.T) {
43
+ for _, tc := range tc.findTests {
44
+ t.Run(tc.name, func(t *testing.T) {
45
+ id, ok := uf.Find(tc.p)
46
+ assert.Equal(t, tc.expectedID, id)
47
+ assert.Equal(t, tc.expectedOK, ok)
48
+ })
49
+ }
50
+ })
51
+
52
+ t.Run("IsConnected", func(t *testing.T) {
53
+ for _, tc := range tc.connectivityTests {
54
+ t.Run(tc.name, func(t *testing.T) {
55
+ assert.Equal(t, tc.expectedIsConnected, uf.IsConnected(tc.p, tc.q))
56
+ })
57
+ }
58
+ })
59
+ }
60
+
61
+ func TestQuickFind(t *testing.T) {
62
+ tests := []unionFindTest{
63
+ {
64
+ name: "OK",
65
+ n: 10,
66
+ unions: [][2]int{
67
+ {-1, -2}, // invalid
68
+ {0, 1},
69
+ {2, 3},
70
+ {2, 4},
71
+ {3, 4}, // alredy connected
72
+ {5, 6},
73
+ {6, 7},
74
+ {8, 7},
75
+ {8, 6}, // alredy connected
76
+ {11, 12}, // invalid
77
+ },
78
+ expectedCount: 4,
79
+ findTests: []findTest{
80
+ {
81
+ name: "Invalid",
82
+ p: -1,
83
+ expectedID: -1,
84
+ expectedOK: false,
85
+ },
86
+ {
87
+ name: "First",
88
+ p: 1,
89
+ expectedID: 1,
90
+ expectedOK: true,
91
+ },
92
+ {
93
+ name: "Second",
94
+ p: 4,
95
+ expectedID: 4,
96
+ expectedOK: true,
97
+ },
98
+ {
99
+ name: "Third",
100
+ p: 8,
101
+ expectedID: 7,
102
+ expectedOK: true,
103
+ },
104
+ {
105
+ name: "Fourth",
106
+ p: 9,
107
+ expectedID: 9,
108
+ expectedOK: true,
109
+ },
110
+ },
111
+ connectivityTests: []connectivityTest{
112
+ {
113
+ name: "Invalid",
114
+ p: -1,
115
+ q: 11,
116
+ expectedIsConnected: false,
117
+ },
118
+ {
119
+ name: "Connected#1",
120
+ p: 0,
121
+ q: 1,
122
+ expectedIsConnected: true,
123
+ },
124
+ {
125
+ name: "Connected#2",
126
+ p: 2,
127
+ q: 4,
128
+ expectedIsConnected: true,
129
+ },
130
+ {
131
+ name: "Connected#3",
132
+ p: 6,
133
+ q: 8,
134
+ expectedIsConnected: true,
135
+ },
136
+ {
137
+ name: "Disconnected#1",
138
+ p: 1,
139
+ q: 3,
140
+ expectedIsConnected: false,
141
+ },
142
+ {
143
+ name: "Disconnected#2",
144
+ p: 3,
145
+ q: 5,
146
+ expectedIsConnected: false,
147
+ },
148
+ {
149
+ name: "Disconnected#3",
150
+ p: 7,
151
+ q: 9,
152
+ expectedIsConnected: false,
153
+ },
154
+ },
155
+ },
156
+ }
157
+
158
+ for _, tc := range tests {
159
+ t.Run(tc.name, func(t *testing.T) {
160
+ uf := NewQuickFind(tc.n)
161
+ runTest(t, tc, uf)
162
+ })
163
+ }
164
+ }
165
+
166
+ func TestQuickUnion(t *testing.T) {
167
+ tests := []unionFindTest{
168
+ {
169
+ name: "OK",
170
+ n: 10,
171
+ unions: [][2]int{
172
+ {-1, -2}, // invalid
173
+ {0, 1},
174
+ {2, 3},
175
+ {2, 4},
176
+ {3, 4}, // alredy connected
177
+ {5, 6},
178
+ {6, 7},
179
+ {8, 7},
180
+ {8, 6}, // alredy connected
181
+ {11, 12}, // invalid
182
+ },
183
+ expectedCount: 4,
184
+ findTests: []findTest{
185
+ {
186
+ name: "Invalid",
187
+ p: -1,
188
+ expectedID: -1,
189
+ expectedOK: false,
190
+ },
191
+ {
192
+ name: "First",
193
+ p: 1,
194
+ expectedID: 1,
195
+ expectedOK: true,
196
+ },
197
+ {
198
+ name: "Second",
199
+ p: 4,
200
+ expectedID: 4,
201
+ expectedOK: true,
202
+ },
203
+ {
204
+ name: "Third",
205
+ p: 8,
206
+ expectedID: 7,
207
+ expectedOK: true,
208
+ },
209
+ {
210
+ name: "Fourth",
211
+ p: 9,
212
+ expectedID: 9,
213
+ expectedOK: true,
214
+ },
215
+ },
216
+ connectivityTests: []connectivityTest{
217
+ {
218
+ name: "Invalid",
219
+ p: -1,
220
+ q: 11,
221
+ expectedIsConnected: false,
222
+ },
223
+ {
224
+ name: "Connected#1",
225
+ p: 0,
226
+ q: 1,
227
+ expectedIsConnected: true,
228
+ },
229
+ {
230
+ name: "Connected#2",
231
+ p: 2,
232
+ q: 4,
233
+ expectedIsConnected: true,
234
+ },
235
+ {
236
+ name: "Connected#3",
237
+ p: 6,
238
+ q: 8,
239
+ expectedIsConnected: true,
240
+ },
241
+ {
242
+ name: "Disconnected#1",
243
+ p: 1,
244
+ q: 3,
245
+ expectedIsConnected: false,
246
+ },
247
+ {
248
+ name: "Disconnected#2",
249
+ p: 3,
250
+ q: 5,
251
+ expectedIsConnected: false,
252
+ },
253
+ {
254
+ name: "Disconnected#3",
255
+ p: 7,
256
+ q: 9,
257
+ expectedIsConnected: false,
258
+ },
259
+ },
260
+ },
261
+ }
262
+
263
+ for _, tc := range tests {
264
+ t.Run(tc.name, func(t *testing.T) {
265
+ uf := NewQuickUnion(tc.n)
266
+ runTest(t, tc, uf)
267
+ })
268
+ }
269
+ }
270
+
271
+ func TestWeightedQuickUnion(t *testing.T) {
272
+ tests := []unionFindTest{
273
+ {
274
+ name: "OK",
275
+ n: 10,
276
+ unions: [][2]int{
277
+ {-1, -2}, // invalid
278
+ {0, 1},
279
+ {2, 3},
280
+ {2, 4},
281
+ {3, 4}, // alredy connected
282
+ {5, 6},
283
+ {6, 7},
284
+ {8, 7},
285
+ {8, 6}, // alredy connected
286
+ {11, 12}, // invalid
287
+ },
288
+ expectedCount: 4,
289
+ findTests: []findTest{
290
+ {
291
+ name: "Invalid",
292
+ p: -1,
293
+ expectedID: -1,
294
+ expectedOK: false,
295
+ },
296
+ {
297
+ name: "First",
298
+ p: 1,
299
+ expectedID: 0,
300
+ expectedOK: true,
301
+ },
302
+ {
303
+ name: "Second",
304
+ p: 4,
305
+ expectedID: 2,
306
+ expectedOK: true,
307
+ },
308
+ {
309
+ name: "Third",
310
+ p: 8,
311
+ expectedID: 5,
312
+ expectedOK: true,
313
+ },
314
+ {
315
+ name: "Fourth",
316
+ p: 9,
317
+ expectedID: 9,
318
+ expectedOK: true,
319
+ },
320
+ },
321
+ connectivityTests: []connectivityTest{
322
+ {
323
+ name: "Invalid",
324
+ p: -1,
325
+ q: 11,
326
+ expectedIsConnected: false,
327
+ },
328
+ {
329
+ name: "Connected#1",
330
+ p: 0,
331
+ q: 1,
332
+ expectedIsConnected: true,
333
+ },
334
+ {
335
+ name: "Connected#2",
336
+ p: 2,
337
+ q: 4,
338
+ expectedIsConnected: true,
339
+ },
340
+ {
341
+ name: "Connected#3",
342
+ p: 6,
343
+ q: 8,
344
+ expectedIsConnected: true,
345
+ },
346
+ {
347
+ name: "Disconnected#1",
348
+ p: 1,
349
+ q: 3,
350
+ expectedIsConnected: false,
351
+ },
352
+ {
353
+ name: "Disconnected#2",
354
+ p: 3,
355
+ q: 5,
356
+ expectedIsConnected: false,
357
+ },
358
+ {
359
+ name: "Disconnected#3",
360
+ p: 7,
361
+ q: 9,
362
+ expectedIsConnected: false,
363
+ },
364
+ },
365
+ },
366
+ }
367
+
368
+ for _, tc := range tests {
369
+ t.Run(tc.name, func(t *testing.T) {
370
+ uf := NewWeightedQuickUnion(tc.n)
371
+ runTest(t, tc, uf)
372
+ })
373
+ }
374
+ }
common/bufferpool/bufferpool.go ADDED
@@ -0,0 +1,29 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ package bufferpool
2
+
3
+ import (
4
+ "bytes"
5
+
6
+ "github.com/metatube-community/metatube-sdk-go/common/pool"
7
+ )
8
+
9
+ type BufferPool struct {
10
+ pool *pool.Pool[*bytes.Buffer]
11
+ }
12
+
13
+ func New(size int) *BufferPool {
14
+ return &BufferPool{
15
+ pool: pool.New(func() *bytes.Buffer {
16
+ return bytes.NewBuffer(make([]byte, 0, size))
17
+ }),
18
+ }
19
+ }
20
+
21
+ func (bp *BufferPool) Get() *bytes.Buffer {
22
+ buf := bp.pool.Get()
23
+ buf.Reset()
24
+ return buf
25
+ }
26
+
27
+ func (bp *BufferPool) Put(b *bytes.Buffer) {
28
+ bp.pool.Put(b)
29
+ }
common/bufferpool/bufferpool_test.go ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ package bufferpool
2
+
3
+ import (
4
+ "strings"
5
+ "testing"
6
+
7
+ "github.com/stretchr/testify/assert"
8
+ )
9
+
10
+ func TestBufferPool(t *testing.T) {
11
+ const size = 1024
12
+ bp := New(size)
13
+ for i := 0; i < 10; i++ {
14
+ buf := bp.Get()
15
+ assert.NotNil(t, buf)
16
+ assert.Equal(t, buf.Len(), 0)
17
+ assert.GreaterOrEqual(t, buf.Cap(), size)
18
+ buf.WriteString(strings.Repeat("\x00", size*2))
19
+ bp.Put(buf)
20
+ }
21
+ }
common/cluster/group.go ADDED
@@ -0,0 +1,42 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ package cluster
2
+
3
+ import (
4
+ "cmp"
5
+
6
+ "github.com/metatube-community/metatube-sdk-go/collection/unionfind"
7
+ )
8
+
9
+ type Group[T Locatable[T, R], R cmp.Ordered] struct {
10
+ Items []T
11
+ }
12
+
13
+ // GroupByDistance partitions a slice of items into proximity-based groups.
14
+ // Any two items whose DistanceTo value is less than or equal to the given
15
+ // threshold are considered to belong to the same group.
16
+ func GroupByDistance[T Locatable[T, R], R cmp.Ordered](items []T, threshold R) []Group[T, R] {
17
+ n := len(items)
18
+ uf := unionfind.NewQuickUnion(n)
19
+
20
+ for i := 0; i < n; i++ {
21
+ for j := i + 1; j < n; j++ {
22
+ if items[i].DistanceTo(items[j]) <= threshold {
23
+ uf.Union(i, j)
24
+ }
25
+ }
26
+ }
27
+
28
+ groupMap := make(map[int][]T)
29
+ for i := 0; i < n; i++ {
30
+ root, _ := uf.Find(i)
31
+ groupMap[root] = append(groupMap[root], items[i])
32
+ }
33
+
34
+ var groups []Group[T, R]
35
+ for _, group := range groupMap {
36
+ groups = append(groups, Group[T, R]{
37
+ Items: group,
38
+ })
39
+ }
40
+
41
+ return groups
42
+ }
common/cluster/group_test.go ADDED
@@ -0,0 +1,56 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ package cluster
2
+
3
+ import (
4
+ "math"
5
+ "testing"
6
+
7
+ "github.com/stretchr/testify/assert"
8
+ )
9
+
10
+ var _ WeightedLocatable[weightedPoint, float64, float64] = (*weightedPoint)(nil)
11
+
12
+ // weightedPoint is a simple 2D point that implements Locatable[weightedPoint].
13
+ type weightedPoint struct {
14
+ X, Y, W float64
15
+ }
16
+
17
+ func (a weightedPoint) DistanceTo(b weightedPoint) float64 {
18
+ dx := a.X - b.X
19
+ dy := a.Y - b.Y
20
+ return math.Hypot(dx, dy)
21
+ }
22
+
23
+ func (a weightedPoint) Weight() float64 {
24
+ return a.W
25
+ }
26
+
27
+ func TestGroupByDistanceAndSort(t *testing.T) {
28
+ points := []weightedPoint{
29
+ {0.10, 0.10, 1.0},
30
+ {0.12, 0.10, 1.0},
31
+ {0.13, 0.12, 2.0},
32
+ {0.90, 0.90, 3.0},
33
+ {0.91, 0.93, 3.0},
34
+ {0.52, 0.90, 5.0},
35
+ }
36
+
37
+ threshold := 0.05
38
+
39
+ // Group points by distance.
40
+ groups := GroupByDistance[weightedPoint, float64](points, threshold)
41
+
42
+ // Assert group count.
43
+ assert.Len(t, groups, 3)
44
+
45
+ // Sort groups by size (descending) and verify.
46
+ SortGroupsBySize(groups)
47
+ assert.Len(t, groups[0].Items, 3)
48
+ assert.Len(t, groups[1].Items, 2)
49
+ assert.Len(t, groups[2].Items, 1)
50
+
51
+ // Sort groups by total weight (descending) and verify.
52
+ SortGroupsByWeight(groups)
53
+ assert.Len(t, groups[0].Items, 2)
54
+ assert.Len(t, groups[1].Items, 1)
55
+ assert.Len(t, groups[2].Items, 3)
56
+ }
common/cluster/locatable.go ADDED
@@ -0,0 +1,22 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ package cluster
2
+
3
+ import (
4
+ "cmp"
5
+ )
6
+
7
+ // Locatable is a generic interface for any type that can
8
+ // measure distance to another value of the same type.
9
+ type Locatable[T any, R cmp.Ordered] interface {
10
+ DistanceTo(T) R
11
+ }
12
+
13
+ // Weighted represents a value that carries a weight.
14
+ type Weighted[W cmp.Ordered] interface {
15
+ Weight() W
16
+ }
17
+
18
+ // WeightedLocatable combines both Locatable and Weighted behaviors.
19
+ type WeightedLocatable[T any, R cmp.Ordered, W cmp.Ordered] interface {
20
+ Locatable[T, R]
21
+ Weighted[W]
22
+ }
common/cluster/sort.go ADDED
@@ -0,0 +1,44 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ package cluster
2
+
3
+ import (
4
+ "cmp"
5
+ "slices"
6
+ "sort"
7
+
8
+ weighted "github.com/metatube-community/metatube-sdk-go/collection/slices"
9
+ )
10
+
11
+ // SortGroupsBySize sorts the provided groups in-place by descending the number of items.
12
+ func SortGroupsBySize[T Locatable[T, R], R cmp.Ordered](groups []Group[T, R]) {
13
+ sort.SliceStable(groups, func(i, j int) bool {
14
+ return len(groups[i].Items) > len(groups[j].Items)
15
+ })
16
+ }
17
+
18
+ // SortGroupsByWeight sorts the provided groups in descending order of total weight.
19
+ func SortGroupsByWeight[T WeightedLocatable[T, R, W], R cmp.Ordered, W cmp.Ordered](groups []Group[T, R]) {
20
+ if len(groups) <= 1 {
21
+ return
22
+ }
23
+
24
+ // group weight calculator.
25
+ weight := func(group Group[T, R]) W {
26
+ var sum W
27
+ for _, item := range group.Items {
28
+ sum += item.Weight()
29
+ }
30
+ return sum
31
+ }
32
+
33
+ // calculate weights for each group.
34
+ weights := slices.Collect(func(yield func(W) bool) {
35
+ for _, group := range groups {
36
+ if !yield(weight(group)) {
37
+ return
38
+ }
39
+ }
40
+ })
41
+
42
+ // weighted stable sort.
43
+ weighted.NewWeightedSlice(groups, weights).SortFunc(sort.Stable)
44
+ }
common/comparer/compare.go ADDED
@@ -0,0 +1,16 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ package comparer
2
+
3
+ import (
4
+ "github.com/adrg/strutil/metrics"
5
+ )
6
+
7
+ // Compare returns the similarity between two strings.
8
+ func Compare(a, b string) float64 {
9
+ m := &metrics.Levenshtein{
10
+ CaseSensitive: false,
11
+ InsertCost: 1,
12
+ DeleteCost: 1,
13
+ ReplaceCost: 2,
14
+ }
15
+ return m.Compare(a, b)
16
+ }
common/comparer/compare_test.go ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ package comparer
2
+
3
+ import "testing"
4
+
5
+ func TestCompare(t *testing.T) {
6
+ for _, unit := range []struct {
7
+ a, b string
8
+ }{
9
+ {"ABP-030", "ABP-030"},
10
+ {"abp-030", "ABP-030"},
11
+ {"ABS-030", "ABP-030"},
12
+ {"AABP-030", "ABP-030"},
13
+ {"KABP-030", "ABP-030"},
14
+ {"ABP-030SP", "ABP-030"},
15
+ {"松下紗栄子", "松下紗栄"},
16
+ {"松下紗栄子", "松下栄子"},
17
+ {"つぼみ", "Bae Bom"},
18
+ {"松下紗栄子", "つぼみ"},
19
+ {"つ", "つぼみ"},
20
+ {"木村夏菜子", "木村夏"},
21
+ {"木村夏菜子", "夏菜子"},
22
+ {"葵", "葵千恵"},
23
+ {"葵", "葵千"},
24
+ {"葵", "葵つかさ"},
25
+ } {
26
+ t.Log(unit.a, unit.b, Compare(unit.a, unit.b))
27
+ }
28
+ }
common/convertor/convert.go ADDED
@@ -0,0 +1,11 @@
 
 
 
 
 
 
 
 
 
 
 
 
1
+ package convertor
2
+
3
+ import (
4
+ "math"
5
+ )
6
+
7
+ // ConvertToCentimeters converts feet and inch to cm.
8
+ func ConvertToCentimeters(feet, inches int) int {
9
+ cm := math.Round(float64(feet*12+inches) * 2.54)
10
+ return int(cm)
11
+ }
common/convertor/convert_test.go ADDED
@@ -0,0 +1,22 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ package convertor
2
+
3
+ import (
4
+ "testing"
5
+
6
+ "github.com/stretchr/testify/assert"
7
+ )
8
+
9
+ func TestConvertToCentimeters(t *testing.T) {
10
+ for _, unit := range []struct {
11
+ ft, in, cm int
12
+ }{
13
+ {5, 5, 165},
14
+ {5, 6, 168},
15
+ {5, 7, 170},
16
+ {5, 8, 173},
17
+ {5, 9, 175},
18
+ {5, 10, 178},
19
+ } {
20
+ assert.Equal(t, unit.cm, ConvertToCentimeters(unit.ft, unit.in))
21
+ }
22
+ }
common/convertor/replace.go ADDED
@@ -0,0 +1,18 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ package convertor
2
+
3
+ import (
4
+ "strings"
5
+ "unicode"
6
+ )
7
+
8
+ // ReplaceSpaceAll removes all spaces in string.
9
+ func ReplaceSpaceAll(s string) string {
10
+ var b strings.Builder
11
+ b.Grow(len(s))
12
+ for _, c := range s {
13
+ if !unicode.IsSpace(c) {
14
+ b.WriteRune(c)
15
+ }
16
+ }
17
+ return b.String()
18
+ }
common/convertor/replace_test.go ADDED
@@ -0,0 +1,19 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ package convertor
2
+
3
+ import (
4
+ "testing"
5
+
6
+ "github.com/stretchr/testify/assert"
7
+ )
8
+
9
+ func TestReplaceSpaceAll(t *testing.T) {
10
+ for _, unit := range []struct {
11
+ origin, expect string
12
+ }{
13
+ {"Hello, world!", "Hello,world!"},
14
+ {"Hello,\tworld!", "Hello,world!"},
15
+ {"\t\tHe\tllo, \tworld! \t", "Hello,world!"},
16
+ } {
17
+ assert.Equal(t, unit.expect, ReplaceSpaceAll(unit.origin))
18
+ }
19
+ }
common/fetch/body.go ADDED
@@ -0,0 +1,25 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ package fetch
2
+
3
+ import (
4
+ "bytes"
5
+ "encoding/json"
6
+ "io"
7
+ "net/url"
8
+ "strings"
9
+ )
10
+
11
+ func WithJSONBody(v any) io.Reader {
12
+ buf := &bytes.Buffer{}
13
+ if err := json.NewEncoder(buf).Encode(v); err != nil {
14
+ panic(err)
15
+ }
16
+ return buf
17
+ }
18
+
19
+ func WithURLEncodedBody(query map[string]string) io.Reader {
20
+ v := &url.Values{}
21
+ for key, value := range query {
22
+ v.Set(key, value)
23
+ }
24
+ return strings.NewReader(v.Encode())
25
+ }
common/fetch/fetch.go ADDED
@@ -0,0 +1,167 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ package fetch
2
+
3
+ import (
4
+ "crypto/tls"
5
+ "io"
6
+ "net/http"
7
+ "net/http/cookiejar"
8
+ "time"
9
+
10
+ "github.com/hashicorp/go-cleanhttp"
11
+ "github.com/hashicorp/go-retryablehttp"
12
+
13
+ "github.com/metatube-community/metatube-sdk-go/common/random"
14
+ "github.com/metatube-community/metatube-sdk-go/errors"
15
+ )
16
+
17
+ var DefaultFetcher = Default(&Config{RandomUserAgent: true})
18
+
19
+ type Config struct {
20
+ // Set User-Agent Header.
21
+ UserAgent string
22
+
23
+ // Set Referer Header.
24
+ Referer string
25
+
26
+ // Enable cookies.
27
+ EnableCookies bool
28
+
29
+ // Use random User-Agent.
30
+ RandomUserAgent bool
31
+
32
+ // Return error when status is not OK.
33
+ RaiseForStatus bool
34
+
35
+ // HTTP Request timeout.
36
+ Timeout time.Duration
37
+
38
+ // Custom HTTP Transport.
39
+ Transport http.RoundTripper
40
+
41
+ // Skip TLS verification. Applies only
42
+ // to *http.Transport based transport.
43
+ SkipVerify bool
44
+ }
45
+
46
+ type Fetcher struct {
47
+ client *http.Client
48
+ config *Config
49
+ }
50
+
51
+ func New(c *http.Client, cfg *Config) *Fetcher {
52
+ if cfg.RandomUserAgent {
53
+ // assign a random user-agent.
54
+ cfg.UserAgent = random.UserAgent()
55
+ }
56
+ if cfg.EnableCookies {
57
+ jar, _ := cookiejar.New(nil)
58
+ c.Jar = jar // assign a cookie jar.
59
+ }
60
+ return &Fetcher{
61
+ client: c,
62
+ config: cfg,
63
+ }
64
+ }
65
+
66
+ func Default(cfg *Config) *Fetcher {
67
+ if cfg == nil /* init if nil */ {
68
+ cfg = new(Config)
69
+ }
70
+ // Enable status check by default.
71
+ cfg.RaiseForStatus = true
72
+ // Enable random UA if not set.
73
+ if cfg.UserAgent == "" {
74
+ cfg.RandomUserAgent = true
75
+ }
76
+ c := &retryablehttp.Client{
77
+ HTTPClient: cleanhttp.DefaultPooledClient(),
78
+ RetryWaitMin: 1 * time.Second,
79
+ RetryWaitMax: 3 * time.Second,
80
+ RetryMax: 3,
81
+ CheckRetry: retryablehttp.DefaultRetryPolicy,
82
+ Backoff: retryablehttp.DefaultBackoff,
83
+ }
84
+ if cfg.Timeout > time.Second {
85
+ c.HTTPClient.Timeout = cfg.Timeout
86
+ }
87
+ if cfg.Transport != nil {
88
+ c.HTTPClient.Transport = cfg.Transport
89
+ }
90
+ if cfg.SkipVerify {
91
+ if transport, ok := c.HTTPClient.Transport.(*http.Transport); ok {
92
+ if transport.TLSClientConfig == nil {
93
+ // init TLS config if is nil.
94
+ transport.TLSClientConfig = &tls.Config{}
95
+ }
96
+ transport.TLSClientConfig.InsecureSkipVerify = true
97
+ }
98
+ }
99
+ return New(c.StandardClient(), cfg)
100
+ }
101
+
102
+ func (f *Fetcher) Fetch(url string) (resp *http.Response, err error) {
103
+ return f.Get(url)
104
+ }
105
+
106
+ func (f *Fetcher) Get(url string, opts ...Option) (resp *http.Response, err error) {
107
+ return f.Request(http.MethodGet, url, nil, opts...)
108
+ }
109
+
110
+ func (f *Fetcher) Post(url string, body io.Reader, opts ...Option) (resp *http.Response, err error) {
111
+ return f.Request(http.MethodPost, url, body, opts...)
112
+ }
113
+
114
+ func (f *Fetcher) Request(method, url string, body io.Reader, opts ...Option) (resp *http.Response, err error) {
115
+ var req *http.Request
116
+ if req, err = http.NewRequest(method, url, body); err != nil {
117
+ return
118
+ }
119
+ c := &Context{
120
+ req: req,
121
+ Config: *f.config, /* clone */
122
+ }
123
+ // compose options.
124
+ var options []Option
125
+ if c.UserAgent != "" {
126
+ options = append(options, WithUserAgent(c.UserAgent))
127
+ }
128
+ if c.Referer != "" {
129
+ options = append(options, WithReferer(c.Referer))
130
+ }
131
+ // apply options.
132
+ for _, option := range append(options, opts...) {
133
+ option.apply(c)
134
+ }
135
+ // make HTTP request.
136
+ if resp, err = f.client.Do(req); err != nil {
137
+ return
138
+ }
139
+ if c.RaiseForStatus && resp.StatusCode != http.StatusOK {
140
+ defer resp.Body.Close()
141
+ return nil, errors.FromCode(resp.StatusCode)
142
+ }
143
+ return
144
+ }
145
+
146
+ func Fetch(url string) (*http.Response, error) {
147
+ return DefaultFetcher.Fetch(url)
148
+ }
149
+
150
+ func Get(url string, opts ...Option) (*http.Response, error) {
151
+ return DefaultFetcher.Get(url, opts...)
152
+ }
153
+
154
+ func Post(url string, body io.Reader, opts ...Option) (*http.Response, error) {
155
+ return DefaultFetcher.Post(url, body, opts...)
156
+ }
157
+
158
+ func Request(method, url string, body io.Reader, opts ...Option) (*http.Response, error) {
159
+ return DefaultFetcher.Request(method, url, body, opts...)
160
+ }
161
+
162
+ var (
163
+ _ = Fetch
164
+ _ = Get
165
+ _ = Post
166
+ _ = Request
167
+ )
common/fetch/option.go ADDED
@@ -0,0 +1,92 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ package fetch
2
+
3
+ import (
4
+ "net/http"
5
+
6
+ "github.com/metatube-community/metatube-sdk-go/common/random"
7
+ )
8
+
9
+ // Context is used for each request.
10
+ type Context struct {
11
+ req *http.Request
12
+ Config
13
+ }
14
+
15
+ type Option func(*Context)
16
+
17
+ func (opt Option) apply(c *Context) { opt(c) }
18
+
19
+ func WithRaiseForStatus(v bool) Option {
20
+ return func(c *Context) { c.RaiseForStatus = v }
21
+ }
22
+
23
+ func WithRequest(fn func(req *http.Request)) Option {
24
+ return func(c *Context) { fn(c.req) }
25
+ }
26
+
27
+ func WithHeader(key, value string) Option {
28
+ return WithRequest(func(req *http.Request) {
29
+ req.Header.Set(key, value)
30
+ })
31
+ }
32
+
33
+ func WithHeaders(headers map[string]string) Option {
34
+ return WithRequest(func(req *http.Request) {
35
+ for key, value := range headers {
36
+ req.Header.Set(key, value)
37
+ }
38
+ })
39
+ }
40
+
41
+ func WithReferer(referer string) Option {
42
+ return WithHeader("Referer", referer)
43
+ }
44
+
45
+ func WithUserAgent(ua string) Option {
46
+ return WithHeader("User-Agent", ua)
47
+ }
48
+
49
+ func WithRandomUserAgent() Option {
50
+ return WithUserAgent(random.UserAgent())
51
+ }
52
+
53
+ func WithAuthorization(token string) Option {
54
+ return WithHeader("Authorization", "Bearer "+token)
55
+ }
56
+
57
+ func WithBasicAuth(username, password string) Option {
58
+ return WithRequest(func(req *http.Request) {
59
+ req.SetBasicAuth(username, password)
60
+ })
61
+ }
62
+
63
+ func WithQuery(key, value string) Option {
64
+ return WithRequest(func(req *http.Request) {
65
+ q := req.URL.Query()
66
+ q.Set(key, value)
67
+ req.URL.RawQuery = q.Encode()
68
+ })
69
+ }
70
+
71
+ func WithQueryMap(query map[string]string) Option {
72
+ return WithRequest(func(req *http.Request) {
73
+ q := req.URL.Query()
74
+ for key, value := range query {
75
+ q.Set(key, value)
76
+ }
77
+ req.URL.RawQuery = q.Encode()
78
+ })
79
+ }
80
+
81
+ func WithQueryPairs(kv ...string) Option {
82
+ return WithRequest(func(req *http.Request) {
83
+ q := req.URL.Query()
84
+ if len(kv)%2 != 0 {
85
+ panic("invalid key-value pairs")
86
+ }
87
+ for i := 0; i < len(kv); i += 2 {
88
+ q.Set(kv[i], kv[i+1])
89
+ }
90
+ req.URL.RawQuery = q.Encode()
91
+ })
92
+ }
common/js/js.go ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ package js
2
+
3
+ import (
4
+ "encoding/json"
5
+ "errors"
6
+
7
+ "github.com/robertkrimen/otto"
8
+ )
9
+
10
+ func UnmarshalObject[T ~string | ~[]byte](jsCode T, objName string, i any) error {
11
+ if len(jsCode) == 0 {
12
+ return errors.New("empty JS code snippet")
13
+ }
14
+
15
+ vm := otto.New()
16
+ v, _ := vm.Run(jsCode)
17
+
18
+ var err error
19
+ if objName != "" {
20
+ v, err = vm.Get(objName)
21
+ if err != nil {
22
+ return err
23
+ }
24
+ }
25
+ b, err := v.MarshalJSON()
26
+ if err != nil {
27
+ return err
28
+ }
29
+ return json.Unmarshal(b, i)
30
+ }
common/js/js_test.go ADDED
@@ -0,0 +1,48 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ package js
2
+
3
+ import (
4
+ "testing"
5
+
6
+ "github.com/stretchr/testify/assert"
7
+ )
8
+
9
+ func TestUnmarshalObject(t *testing.T) {
10
+ {
11
+ data := struct {
12
+ MovieSeq string `json:"movie_seq"`
13
+ Page int `json:"page"`
14
+ Lang string `json:"lang"`
15
+ ProviderName string `json:"provider_name"`
16
+ }{}
17
+ jsCode := `var object = {movie_seq: '2969', page: 1, type: 'monthly', provider_name: 'heyzo', lang: 'ja'};
18
+ reviews_get(object);`
19
+ objName := "object"
20
+ err := UnmarshalObject(jsCode, objName, &data)
21
+ assert.NoError(t, err)
22
+ assert.Equal(t, "2969", data.MovieSeq)
23
+ assert.Equal(t, 1, data.Page)
24
+ assert.Equal(t, "ja", data.Lang)
25
+ }
26
+
27
+ {
28
+ data := struct {
29
+ Comments []struct {
30
+ Username string `json:"user_name"`
31
+ } `json:"comments"`
32
+ }{}
33
+ jsCode := `var reviews = {"comments":[{"star":"\u2605?","user_name":"HEY","date":"2023-02-11 18:38:18","comment":"\u30bf\u30c4\u541b\u3001\u3044\u3064\u3082\u306f\u3082\u3063\u3068\u7a4d\u6975\u7684\u306a\u611f\u3058\u306a\u306e\u306b\u4eca\u56de\u306f\u89aa\u53cb\u306e\u5f7c\u5973\u3068\u7d61\u3080\u3068\u3044\u3046\u5f79\u67c4\u8a2d\u5b9a\u304b\u3089\u306a\u306e\u304b\u7a4d\u6975\u6027\u306b\u6b20\u3051\u307e\u3059\u306d\u3002\u306a\u307f\u3061\u3083\u3093\u306f\u7a4d\u6975\u7684\u306a\u5f79\u306a\u306e\u3067\u3001\u306a\u307f\u3061\u3083\u3093\u306b\u305f\u3059\u3051\u3063\u308c\u3066\u3044\u308b\u90e8\u5206\u304c\u5927\u304d\u3044\u304b\u306a\uff1f\u306a\u307f\u3061\u3083\u3093\u306e\u304a\u304b\u3052\u3067\u6e80\u70b9\u3067\u3059\u203c\u3067\u3082\u9732\u51fa\u597d\u304d\u306a\u3089\u3001\u30bf\u30c4\u541b\u304c\u8208\u596e\u3059\u308b\u3088\u3046\u306b\u3082\u3063\u3068\u30a8\u30c3\u30c1\u306a\u6311\u767a\u3057\u306a\u3044\u3068\u306d\u3002","comment_res":"","point_flag":0,"monthly_point_flag":0,"hash":"f0cbd8148980aed6709e52907a9c8b8b","base64":"8MvYFImArt","id":"53627","eng":"0","score":{"overall":"5","quality":"5","content":"5","actress":"5","play":"5","price":"5"},"vote":{"yes":"0","no":"0"}},{"star":"\u2605?","user_name":"\u98a8\u96f2\u5150","date":"2023-01-05 11:45:15","comment":"\u306a\u3093\u304b\u7537\u512a\u304b\u3089\u306e\u8cac\u3081\u3067\u76db\u308a\u4e0a\u304c\u308a\u306b\u6b20\u3051\u308b\u306e\u304c\u6b98\u5ff5\u3002\u5973\u512a\u3055\u3093\u304c\u305f\u3060\u4e00\u751f\u61f8\u547d\u9811\u5f35\u3063\u3066\u308b\u3063\u3066\u611f\u3058\u3002\u3042\u3068\u3001\u3069\u3053\u304c\u300c\u9732\u51fa\u300d\u3060\u3088\uff57\uff57","comment_res":"","point_flag":0,"monthly_point_flag":0,"hash":"ce115f66a3a91e055804734830ba43c8","base64":"zhFfZqOpHg","id":"53477","eng":"0","score":{"overall":"4","quality":"5","content":"5","actress":"5","play":"5","price":"5"},"vote":{"yes":"0","no":"0"}},{"star":"\u2605?","user_name":"\u4e16\u754c\uff11\u4f4d\u306e\u7537","date":"2023-01-03 09:06:15","comment":"\u3053\u308c\u306f\u6587\u53e5\u306a\u3057\u306e\u826f\u4f5c\u3067\u3059\u306d\u3002\u3053\u3093\u306a\u306b\u30a8\u30ed\u304f\u8feb\u3063\u3066\u304f\u308c\u305f\u3089\u3059\u3050\u6483\u6c88\u3057\u305d\u3046\u3067\u3059\u3002","comment_res":"","point_flag":0,"monthly_point_flag":0,"hash":"385f9d684324bec37663651c02b8010d","base64":"OF+daEMkvs","id":"53463","eng":"0","score":{"overall":"5","quality":"5","content":"5","actress":"5","play":"5","price":"5"},"vote":{"yes":"0","no":"0"}},{"star":"\u2605?","user_name":"\u30aa\u30b8\u30b5\u30f3","date":"2023-01-01 13:07:18","comment":"\u5b89\u5ba4\u306a\u307f\u3061\u3083\u3093\u304b\u308f\u3044\u304f\u3066\u5927\u304d\u3081\u306e\u30aa\u30c3\u30d1\u30a4\u306b\u30d7\u30ea\u30f3\u3068\u3057\u305f\u304a\u5c3b\u306e\u30b9\u30ec\u30f3\u30c0\u30fc\u306a\u8eab\u4f53\u3067\u4e00\u672c\u7b4b\u306e\u7f8e\u30de\u30f3\u304c\u3044\u3044\u3067\u3059\u306d\u3002\u4f8b\u3048\u89aa\u53cb\u306e\u5f7c\u5973\u3067\u3042\u3063\u305f\u3068\u3057\u3066\u3082\u306a\u307f\u3061\u3083\u3093\u306e\u65b9\u304b\u3089\u7a4d\u6975\u7684\u306b\u8feb\u3089\u308c\u305f\u3089\u30bb\u30c3\u30af\u30b9\u3057\u306a\u3044\u8a33\u306b\u306f\u3044\u304d\u307e\u305b\u3093\u306d\u3002\u30bf\u30c4\u304f\u3093\u3068\u306e\u604b\u4eba\u540c\u58eb\u306e\u3088\u3046\u306a\u7d61\u307f\u306e\u30bb\u30c3\u30af\u30b9\u3092\u3057\u3066\u3044\u308b\u306a\u307f\u3061\u3083\u3093\u306f\u7f8e\u30dc\u30c7\u30a3\u3082\u76f8\u307e\u3063\u3066\u3068\u3066\u3082\u30ad\u30ec\u30a4\u3067\u7d20\u6575\u3067\u3057\u305f\u3002\u6587\u53e5\u306a\u3057\u306e\u6e80\u70b9\u3067\u3059\u3002","comment_res":"","point_flag":0,"monthly_point_flag":0,"hash":"91cf09fbcca9303e8a4d34775f830fed","base64":"kc8J+8ypMD","id":"53455","eng":"0","score":{"overall":"5","quality":"5","content":"5","actress":"5","play":"5","price":"5"},"vote":{"yes":"0","no":"0"}},{"star":"\u2605?","user_name":"\u6c41\u5973\u512a\u547d","date":"2023-01-01 12:42:16","comment":"\u9a0e\u4e57\u4f4d\u3067\u306e\u7d20\u6674\u3089\u3057\u3044\u8170\u632f\u308a\u3002\u30bd\u30d5\u30a1\u30fc\u3067\uff13\uff10\u5206\u4f4d\u3001\u80cc\u9762\u9a0e\u4e57\u3057\u3066\u308b\u5f71\u50cf\u304c\u3001\u898b\u305f\u3044\u3067\u3059\u3002","comment_res":"","point_flag":0,"monthly_point_flag":0,"hash":"215632e4acfffdabadfc20bad4823cbc","base64":"IVYy5Kz\/\/a","id":"53454","eng":"0","score":{"overall":"5","quality":"5","content":"5","actress":"5","play":"5","price":"5"},"vote":{"yes":"0","no":"0"}},{"star":"\u2605?","user_name":"tsuruman","date":"2023-01-01 07:52:02","comment":"\u5de6\u80a9\u306e\u30bf\u30c8\u30a5\u306f\u3082\u3046\u5c11\u3057\u4e0a\u624b\u304f\u6d88\u305b\u306a\u3044\u306e\u304b\u3057\u3089\u3002\u3053\u308c\u304c\u76ee\u306b\u3064\u3044\u3066\u3001\u3069\u3046\u3057\u3066\u3082\u5b89\u3063\u307d\u3044\u5973\u306b\u898b\u3048\u3061\u3083\u3046\u3002","comment_res":"","point_flag":0,"monthly_point_flag":0,"hash":"bfe6cd26fc89ffbb479d6b4087dd7a2a","base64":"v+bNJvyJ\/7","id":"53453","eng":"0","score":{"overall":"4","quality":"5","content":"5","actress":"5","play":"5","price":"5"},"vote":{"yes":"0","no":"0"}},{"star":"\u2605?","user_name":"nanchun","date":"2023-01-01 02:17:23","comment":"just amazing","comment_res":"","point_flag":0,"monthly_point_flag":0,"hash":"93157bc26dcea2d1a16f28e7816dc07a","base64":"kxV7wm3Oot","id":"53452","eng":"0","score":{"overall":"5","quality":"5","content":"5","actress":"5","play":"5","price":"5"},"vote":{"yes":"0","no":"0"}}],"pages":{"curr":"1","movie_seq":"2969","showall":0,"prev":0,"next":0,"first":0,"last":0,"lang":"ja","n":[1]},"count_ja":7,"count_en":1};`
34
+ objName := "reviews"
35
+ err := UnmarshalObject(jsCode, objName, &data)
36
+ assert.NoError(t, err)
37
+ assert.Equal(t, "HEY", data.Comments[0].Username)
38
+ }
39
+
40
+ {
41
+ data := ""
42
+ jsCode := []byte(`var abc = 'hello'`)
43
+ objName := "abc"
44
+ err := UnmarshalObject(jsCode, objName, &data)
45
+ assert.NoError(t, err)
46
+ assert.Equal(t, "hello", data)
47
+ }
48
+ }