3508 Commits

Author SHA1 Message Date
fuxiaohei
dff0d4b907 feat(runner): update edit.html 2022-11-25 17:48:45 +08:00
fuxiaohei
206b2a104e feat(runner): begin runner view ui 2022-11-25 17:48:44 +08:00
Jason Song
79966763c6 feat: distribute tasks to runner 2022-11-25 17:48:44 +08:00
Bo-Yi.Wu
6f18f76577 chore(runner): update runner status using proto define
Signed-off-by: Bo-Yi.Wu <appleboy.tw@gmail.com>
2022-11-25 17:48:44 +08:00
Jason Song
530faf7227 feat: assign task to runner 2022-11-25 17:48:44 +08:00
Bo-Yi.Wu
ade88b3a7b chore(runner): register new runner
Signed-off-by: Bo-Yi.Wu <appleboy.tw@gmail.com>
2022-11-25 17:48:44 +08:00
Bo-Yi.Wu
0993c40c4f chore(runner): create runner token table
Signed-off-by: Bo-Yi.Wu <appleboy.tw@gmail.com>
2022-11-25 17:48:44 +08:00
Bo-Yi.Wu
4dd135580f chore(runner): update to latest proto format.
Signed-off-by: Bo-Yi.Wu <appleboy.tw@gmail.com>
2022-11-25 17:48:44 +08:00
Jason Song
f355188e12 feat: parse workflows 2022-11-25 17:48:44 +08:00
Jason Song
905bce6cb6 feat: models for runner task 2022-11-25 17:48:44 +08:00
wxiaoguang
5f0cb6b9e5 fix merge 2022-11-25 17:48:44 +08:00
Bo-Yi.Wu
fb738c4192 chore(runner): support fetch detail stage data
Signed-off-by: Bo-Yi.Wu <appleboy.tw@gmail.com>
2022-11-25 17:48:44 +08:00
Lunny Xiao
db02603882 Fix bug 2022-11-25 17:48:44 +08:00
Bo-Yi.Wu
8255915350 chore: move core package to top folder.
Signed-off-by: Bo-Yi.Wu <appleboy.tw@gmail.com>
2022-11-25 17:48:44 +08:00
Bo-Yi.Wu
1148903d79 refactor: move build status to core package 2022-11-25 17:48:44 +08:00
Bo-Yi.Wu
ea0cf8515e chore(gRPC): handle requesut for stage data
Signed-off-by: Bo-Yi.Wu <appleboy.tw@gmail.com>
2022-11-25 17:48:44 +08:00
Bo-Yi.Wu
68d416bd1b chore(gRPC): initial request method
Signed-off-by: Bo-Yi.Wu <appleboy.tw@gmail.com>
2022-11-25 17:48:44 +08:00
Bo-Yi.Wu
e07d416366 chore(model): create build, stage, step and logs table
Signed-off-by: Bo-Yi.Wu <appleboy.tw@gmail.com>
2022-11-25 17:48:44 +08:00
Bo-Yi.Wu
e05f224d19 chore(runner): support register
Signed-off-by: Bo-Yi.Wu <appleboy.tw@gmail.com>
2022-11-25 17:48:44 +08:00
wxiaoguang
92d15afd18 database-filesystem, demo log tailing in term 2022-11-25 17:48:43 +08:00
Lunny Xiao
931d8c2c21 add runners management ui 2022-11-25 17:48:43 +08:00
Lunny Xiao
a1bd3a8932 Fix build 2022-11-25 17:48:43 +08:00
Lunny Xiao
2c4f6fd42f add most tables 2022-11-25 17:48:43 +08:00
Lunny Xiao
5a479bb034 Add builds UI 2022-11-25 17:48:43 +08:00
Lunny Xiao
7732392a96 Add bots 2022-11-25 17:48:33 +08:00
Jason Song
702f4b521e
Add index for access_token (#21908)
The `token_last_eight` field has been used in `GetAccessTokenBySHA `:
2022-11-23 20:49:41 -06:00
Xinyu Zhou
e483ec3a00
Fix vertical align of committer avatar rendered by email address (#21884)
Committer avatar rendered by `func AvatarByEmail` are not vertical align
as `func Avatar` does.

- Replace literals `ui avatar` and `ui avatar vm` with the constant
`DefaultAvatarClass`
2022-11-23 15:57:37 -06:00
zeripath
787f6c3227
Ensure that Webhook tasks are not double delivered (#21558)
When re-retrieving hook tasks from the DB double check if they have not
been delivered in the meantime. Further ensure that tasks are marked as
delivered when they are being delivered.

In addition:
* Improve the error reporting and make sure that the webhook task
population script runs in a separate goroutine.
* Only get hook task IDs out of the DB instead of the whole task when
repopulating the queue
* When repopulating the queue make the DB request paged

Ref #17940 

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-11-23 22:10:04 +08:00
zeripath
4c00d8f916
Move migration test fixtures to the correct directories (#21901)
Unfortunately #21549 changed the name of Testcases without changing
their associated fixture directories.

Fix #21854

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-11-23 18:24:55 +08:00
KN4CK3R
32db62515f
Add package registry cleanup rules (#21658)
Fixes #20514
Fixes #20766
Fixes #20631

This PR adds Cleanup Rules for the package registry. This allows to
delete unneeded packages automatically. Cleanup rules can be set up from
the user or org settings.
Please have a look at the documentation because I'm not a native english
speaker.

Rule Form

![grafik](https://user-images.githubusercontent.com/1666336/199330792-c13918a6-e196-4e71-9f53-18554515edca.png)

Rule List

![grafik](https://user-images.githubusercontent.com/1666336/199331261-5f6878e8-a80c-4985-800d-ebb3524b1a8d.png)

Rule Preview

![grafik](https://user-images.githubusercontent.com/1666336/199330917-c95e4017-cf64-4142-a3e4-af18c4f127c3.png)

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-11-20 16:08:38 +02:00
KN4CK3R
044c754ea5
Add context.Context to more methods (#21546)
This PR adds a context parameter to a bunch of methods. Some helper
`xxxCtx()` methods got replaced with the normal name now.

Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-11-19 16:12:33 +08:00
Gusted
20385b52a3
Prevent dangling user redirects (#21856)
- It's possible that the `user_redirect` table contains a user id that
no longer exists.
- Delete a user redirect upon deleting the user.
- Add a check for these dangling user redirects to check-db-consistency.
2022-11-18 22:23:34 +08:00
KN4CK3R
88d5275614
Do not allow Ghost access to limited visible user/org (#21849)
The Ghost user should not be allowed to have access to a limited visible
user/org.

Co-authored-by: Lauris BH <lauris@nix.lv>
2022-11-17 14:29:33 -05:00
zeripath
c772934ff6
Adjust gitea doctor --run storages to check all storage types (#21785)
The doctor check `storages` currently only checks the attachment
storage. This PR adds some basic garbage collection functionality for
the other types of storage.

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-11-15 16:08:59 +08:00
Lunny Xiao
34283a74e8
Allow detect whether it's in a database transaction for a context.Context (#21756)
Fix #19513

This PR introduce a new db method `InTransaction(context.Context)`,
and also builtin check on `db.TxContext` and `db.WithTx`.
There is also a new method `db.AutoTx` has been introduced but could be used by other PRs.

`WithTx` will always open a new transaction, if a transaction exist in context, return an error.
`AutoTx` will try to open a new transaction if no transaction exist in context.
That means it will always enter a transaction if there is no error.

Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: 6543 <6543@obermui.de>
2022-11-12 21:18:50 +01:00
wxiaoguang
92525ddffd
Init git module before database migration (#21764)
Close #21761

Some database migrations depend on the git module.
2022-11-10 14:22:39 +00:00
Lunny Xiao
385462d36c
Fix dashboard ignored system setting cache (#21621)
This is a performance regression from #18058

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Andrew Thornton <art27@cantab.net>
2022-11-10 14:43:53 +08:00
KN4CK3R
20674dd05d
Add package registry quota limits (#21584)
Related #20471

This PR adds global quota limits for the package registry. Settings for
individual users/orgs can be added in a seperate PR using the settings
table.

Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-11-09 14:34:27 +08:00
Lunny Xiao
91c7a3e66f
Fix tests on migrations (#21705) 2022-11-08 12:07:46 +08:00
wxiaoguang
2900dc90a7
Improve valid user name check (#20136)
Close https://github.com/go-gitea/gitea/issues/21640

Before: Gitea can create users like ".xxx" or "x..y", which is not
ideal, it's already a consensus that dot filenames have special
meanings, and `a..b` is a confusing name when doing cross repo compare.

After: stricter

Co-authored-by: Jason Song <i@wolfogre.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: delvh <dev.lh@web.de>
2022-11-04 17:04:08 +08:00
oliverpool
b6e81357bd
Add Webhook authorization header (#20926)
_This is a different approach to #20267, I took the liberty of adapting
some parts, see below_

## Context

In some cases, a weebhook endpoint requires some kind of authentication.
The usual way is by sending a static `Authorization` header, with a
given token. For instance:

- Matrix expects a `Bearer <token>` (already implemented, by storing the
header cleartext in the metadata - which is buggy on retry #19872)
- TeamCity #18667
- Gitea instances #20267
- SourceHut https://man.sr.ht/graphql.md#authentication-strategies (this
is my actual personal need :)

## Proposed solution

Add a dedicated encrypt column to the webhook table (instead of storing
it as meta as proposed in #20267), so that it gets available for all
present and future hook types (especially the custom ones #19307).

This would also solve the buggy matrix retry #19872.

As a first step, I would recommend focusing on the backend logic and
improve the frontend at a later stage. For now the UI is a simple
`Authorization` field (which could be later customized with `Bearer` and
`Basic` switches):


![2022-08-23-142911](https://user-images.githubusercontent.com/3864879/186162483-5b721504-eef5-4932-812e-eb96a68494cc.png)

The header name is hard-coded, since I couldn't fine any usecase
justifying otherwise.

## Questions

- What do you think of this approach? @justusbunsi @Gusted @silverwind 
- ~~How are the migrations generated? Do I have to manually create a new
file, or is there a command for that?~~
- ~~I started adding it to the API: should I complete it or should I
drop it? (I don't know how much the API is actually used)~~

## Done as well:

- add a migration for the existing matrix webhooks and remove the
`Authorization` logic there


_Closes #19872_

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Gusted <williamzijl7@hotmail.com>
Co-authored-by: delvh <dev.lh@web.de>
2022-11-03 20:23:20 +02:00
kolaente
085f717529
feat: notify doers of a merge when automerging (#21553)
I found myself wondering whether a PR I scheduled for automerge was
actually merged. It was, but I didn't receive a mail notification for it
- that makes sense considering I am the doer and usually don't want to
receive such notifications. But ideally I want to receive a notification
when a PR was merged because I scheduled it for automerge.

This PR implements exactly that.

The implementation works, but I wonder if there's a way to avoid passing
the "This PR was automerged" state down so much. I tried solving this
via the database (checking if there's an automerge scheduled for this PR
when sending the notification) but that did not work reliably, probably
because sending the notification happens async and the entry might have
already been deleted. My implementation might be the most
straightforward but maybe not the most elegant.

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-11-03 23:49:00 +08:00
KN4CK3R
f8f859b8fa
Alter package_version.metadata_json to LONGTEXT (#21667) 2022-11-03 15:28:46 +08:00
Lunny Xiao
e72acd5e5b
Split migrations folder (#21549)
There are too many files in `models/migrations` folder so that I split
them into sub folders.
2022-11-02 16:54:36 +08:00
Lunny Xiao
9a70a12a34
Merge db.Iterate and IterateObjects (#21641)
These two functions are similiar, merge them.
2022-10-31 23:51:14 +08:00
Lunny Xiao
f337c32e86
Add index for hook_task table (#21545)
Since `hook_id` and `uuid` will become a search condition column. It's
better to add some index for them.
2022-10-28 13:05:39 +02:00
Lunny Xiao
5e8e3ecbeb
Fix issues count bug (#21557)
fix #19349 , #19505

Co-authored-by: delvh <dev.lh@web.de>
2022-10-25 14:47:46 +02:00
delvh
0ebb45cfe7
Replace all instances of fmt.Errorf(%v) with fmt.Errorf(%w) (#21551)
Found using
`find . -type f -name '*.go' -print -exec vim {} -c
':%s/fmt\.Errorf(\(.*\)%v\(.*\)err/fmt.Errorf(\1%w\2err/g' -c ':wq' \;`

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-10-24 20:29:17 +01:00
M Hickford
191a74d622
Record OAuth client type at registration (#21316)
The OAuth spec [defines two types of
client](https://datatracker.ietf.org/doc/html/rfc6749#section-2.1),
confidential and public. Previously Gitea assumed all clients to be
confidential.

> OAuth defines two client types, based on their ability to authenticate
securely with the authorization server (i.e., ability to
>   maintain the confidentiality of their client credentials):
>
>   confidential
> Clients capable of maintaining the confidentiality of their
credentials (e.g., client implemented on a secure server with
> restricted access to the client credentials), or capable of secure
client authentication using other means.
>
>   **public
> Clients incapable of maintaining the confidentiality of their
credentials (e.g., clients executing on the device used by the resource
owner, such as an installed native application or a web browser-based
application), and incapable of secure client authentication via any
other means.**
>
> The client type designation is based on the authorization server's
definition of secure authentication and its acceptable exposure levels
of client credentials. The authorization server SHOULD NOT make
assumptions about the client type.

 https://datatracker.ietf.org/doc/html/rfc8252#section-8.4

> Authorization servers MUST record the client type in the client
registration details in order to identify and process requests
accordingly.

Require PKCE for public clients:
https://datatracker.ietf.org/doc/html/rfc8252#section-8.1

> Authorization servers SHOULD reject authorization requests from native
apps that don't use PKCE by returning an error message

Fixes #21299

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-10-24 15:59:24 +08:00
wxiaoguang
dcd9fc7ee8
Refactor git command arguments and make all arguments to be safe to be used (#21535)
Follow #21464

Make all git command arguments strictly safe. Most changes are one-to-one replacing, keep all existing logic.
2022-10-23 22:44:45 +08:00