222 Commits

Author SHA1 Message Date
TheFox0x7
ee3c82f874
Enable addtional linters (#34085)
enable mirror, usestdlibbars and perfsprint 
part of: https://github.com/go-gitea/gitea/issues/34083

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-04-01 10:14:01 +00:00
TheFox0x7
0fde8ecd55
Enable testifylint rules (#34075)
enable testifylint rules disabled in:
https://github.com/go-gitea/gitea/pull/34054
2025-03-31 01:53:48 -04:00
TheFox0x7
2a59dfbd47
enable staticcheck QFxxxx rules (#34064) 2025-03-29 17:32:28 -04:00
Lunny Xiao
c1b9ecca34
Decouple Batch from git.Repository to simplify usage without requiring the creation of a Repository struct. (#34001)
No logic change
2025-03-27 17:35:48 +00:00
wxiaoguang
8745129c9c
Fix incorrect code search indexer options (#33992)
Fix #33798

Co-authored-by: Giteabot <teabot@gitea.io>
2025-03-24 17:18:21 +01:00
charles
b956cee06f
Fix file name could not be searched if the file was not a text file when using the Bleve indexer (#33959)
Close #33828

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-03-21 19:00:02 +00:00
Andreas Svanberg
0da7318cf3
Allow filtering issues by any assignee (#33343)
This is the opposite of the "No assignee" filter, it will match all
issues that have at least one assignee.

Before
![Before
change](https://github.com/user-attachments/assets/4aea194b-9add-4a84-8d6b-61bfd8d9e58e)

After
![After change with any
filter](https://github.com/user-attachments/assets/99f1205d-ba9f-4a0a-a60b-cc1a0c0823fe)

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-03-21 04:25:36 +00:00
Lunny Xiao
c88e71c1d2
Refactor functions to reduce repopath expose (#33892) 2025-03-16 03:14:56 +00:00
wxiaoguang
a0e0a30d23
Make SearchMode have default value and add comments (#33863)
* Make `SearchMode` have default value if it is empty
* Add some comments for the "match" queries
* Fix a copy-paste mistake in `buildMatchQuery` (`db.go`)
* Add missing `q.Analyzer = repoIndexerAnalyzer`, it is in old code,
although I do not see real difference ....
2025-03-14 18:06:31 +00:00
wxiaoguang
403775e74e
Improve issue & code search (#33860)
Each "indexer" should provide the "search modes" they support by
themselves. And we need to remove the "fuzzy" search for code.
2025-03-13 11:07:48 +08:00
TheFox0x7
ae3a18e01a
Remove context from git struct (#33793)
Argument is moved from struct init in command run, which lets us remove
context from struct.
2025-03-04 11:56:11 -08:00
TheFox0x7
cc1fdc84ca
Use test context in tests and new loop system in benchmarks (#33648)
Replace all contexts in tests with go1.24 t.Context()

---------

Co-authored-by: Giteabot <teabot@gitea.io>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-02-20 09:57:40 +00:00
Lunny Xiao
69de5a65c2
Fix project issues list and counting (#33594)
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-02-17 05:14:56 +00:00
Darren Hoo
950abfe8ee
enable literal string for code search (#33590)
Close: #33588

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Giteabot <teabot@gitea.io>
2025-02-16 11:28:06 +01:00
Lunny Xiao
6999651b6d
Fix unnecessary comment when moving issue on the same project column (#33496)
Fix #33482
2025-02-05 11:51:10 +08:00
TheFox0x7
2a02734f93
Refactor older tests to use testify (#33140)
Refactor checks to use assert/require
Use require.Eventually for waiting in elastic and meilisearch tests
Use require to exit early instead of assert
2025-01-09 09:21:47 +08:00
wxiaoguang
45973a100b
Fix bleve fuzziness search (#33078)
Close #31565
2025-01-03 00:32:02 +08:00
TheFox0x7
33e8e82c4b
Enable tenv and testifylint rules (#32852)
Enables tenv and testifylint linters
closes: https://github.com/go-gitea/gitea/issues/32842
2024-12-15 10:41:29 +00:00
silverwind
1cfb718976
Update golangci-lint to v1.62.2, fix issues (#32845)
Update it and fix new issues related to `redefines-builtin-id`
2024-12-15 02:31:07 +00:00
yp05327
d43620e7bc
Add is_archived option for issue indexer (#32735)
Try to fix #32697

Reason:
`is_archived` is already defined in the query options, but it is not
implemented in the indexer.
2024-12-12 07:33:31 +08:00
wxiaoguang
e619384098
Add label/author/assignee filters to the user/org home issue list (#32779)
Replace #26661, fix #25979

Not perfect, but usable and much better than before. Since it is quite
complex, I am not quite sure whether there would be any regression, if
any, I will fix in first time.

I have tested the related pages many times: issue list, milestone issue
list, project view, user issue list, org issue list.
2024-12-11 06:33:24 +00:00
wxiaoguang
44909f6e2c
Fix markup render regression and fix some tests (#32640)
Fix #32639,
https://github.com/go-gitea/gitea/issues/32608#issuecomment-2497918210

By the way, fix some incorrect SQLs (use single quote but not double
quote)
2024-11-26 03:04:55 +08:00
Rowan Bohde
68731c07c5
Reduce integration test overhead (#32475)
In profiling integration tests, I found a couple places where per-test
overhead could be reduced:

* Avoiding disk IO by synchronizing instead of deleting & copying test
Git repository data. This saves ~100ms per test on my machine
* When flushing queues in `PrintCurrentTest`, invoke `FlushWithContext`
in a parallel.

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-11-14 19:28:46 +00:00
Bruno Sofiato
f64fbd9b74
Updated tokenizer to better matching when search for code snippets (#32261)
This PR improves the accuracy of Gitea's code search. 

Currently, Gitea does not consider statements such as
`onsole.log("hello")` as hits when the user searches for `log`. The
culprit is how both ES and Bleve are tokenizing the file contents (in
both cases, `console.log` is a whole token).

In ES' case, we changed the tokenizer to
[simple_pattern_split](https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-simplepatternsplit-tokenizer.html#:~:text=The%20simple_pattern_split%20tokenizer%20uses%20a,the%20tokenization%20is%20generally%20faster.).
In such a case, tokens are words formed by digits and letters. In
Bleve's case, it employs a
[letter](https://blevesearch.com/docs/Tokenizers/) tokenizer.

Resolves #32220

---------

Signed-off-by: Bruno Sofiato <bruno.sofiato@gmail.com>
2024-11-06 20:51:20 +00:00
wxiaoguang
5e6523aa57
Update go dependencies (#32389) 2024-10-31 12:05:54 +00:00
Bruno Sofiato
900ac62251
Allow code search by filename (#32210)
This is a large and complex PR, so let me explain in detail its changes.

First, I had to create new index mappings for Bleve and ElasticSerach as
the current ones do not support search by filename. This requires Gitea
to recreate the code search indexes (I do not know if this is a breaking
change, but I feel it deserves a heads-up).

I've used [this
approach](https://www.elastic.co/guide/en/elasticsearch/reference/7.17/analysis-pathhierarchy-tokenizer.html)
to model the filename index. It allows us to efficiently search for both
the full path and the name of a file. Bleve, however, does not support
this out-of-box, so I had to code a brand new [token
filter](https://blevesearch.com/docs/Token-Filters/) to generate the
search terms.

I also did an overhaul in the `indexer_test.go` file. It now asserts the
order of the expected results (this is important since matches based on
the name of a file are more relevant than those based on its content).
I've added new test scenarios that deal with searching by filename. They
use a new repo included in the Gitea fixture.

The screenshot below depicts how Gitea shows the search results. It
shows results based on content in the same way as the current version
does. In matches based on the filename, the first seven lines of the
file contents are shown (BTW, this is how GitHub does it).


![image](https://github.com/user-attachments/assets/9d938d86-1a8d-4f89-8644-1921a473e858)

Resolves #32096

---------

Signed-off-by: Bruno Sofiato <bruno.sofiato@gmail.com>
2024-10-11 23:35:04 +00:00
Bruno Sofiato
d266d190bd
Fixed race condition when deleting documents by repoId in ElasticSearch (#32185)
Resolves #32184

---------

Signed-off-by: Bruno Sofiato <bruno.sofiato@gmail.com>
2024-10-03 16:03:36 +00:00
Bruno Sofiato
99d0510cb6
Change the code search to sort results by relevance (#32134)
Resolves #32129

Signed-off-by: Bruno Sofiato <bruno.sofiato@gmail.com>
2024-09-28 20:13:55 +00:00
Lunny Xiao
286ede47ad
Fix index too many file names bug (#31903)
Try to fix #31884
Fix #28584
2024-09-01 05:57:31 +00:00
Lunny Xiao
c03baab678
Refactor the usage of batch catfile (#31754)
When opening a repository, it will call `ensureValidRepository` and also
`CatFileBatch`. But sometimes these will not be used until repository
closed. So it's a waste of CPU to invoke 3 times git command for every
open repository.

This PR removed all of these from `OpenRepository` but only kept
checking whether the folder exists. When a batch is necessary, the
necessary functions will be invoked.
2024-08-20 17:04:57 +00:00
Kemal Zebari
c0b5a843ba
Properly filter issue list given no assignees filter (#31522)
Quick fix #31520. This issue is related to #31337.
2024-07-23 18:36:32 +00:00
Carsten Klein
3571b7e3dd
Allow searching issues by ID (#31479)
When you are entering a number in the issue search, you likely want the
issue with the given ID (code internal concept: issue index).
As such, when a number is detected, the issue with the corresponding ID
will now be added to the results.

Fixes #4479

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-07-17 00:49:05 +02:00
Lunny Xiao
e4abaff7ff
Fix bug filtering issues which have no project (#31337)
Fix #31327
This is a quick patch to fix the bug.
Some parameters are using 0, some are using -1. I think it needs a
refactor to keep consistent. But that will be another PR.
2024-06-14 02:31:07 +00:00
Lunny Xiao
98751108b1
Rename project board -> column to make the UI less confusing (#30170)
This PR split the `Board` into two parts. One is the struct has been
renamed to `Column` and the second we have a `Template Type`.

But to make it easier to review, this PR will not change the database
schemas, they are just renames. The database schema changes could be in
future PRs.

---------

Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: yp05327 <576951401@qq.com>
2024-05-27 08:59:54 +00:00
wxiaoguang
6f7cd94a02
Fix bleve fuzziness (#30799)
Fix #30797
Fix #30317
2024-05-01 15:32:52 +03:00
silverwind
610802df85
Fix tautological conditions (#30735)
As discovered by https://github.com/go-gitea/gitea/pull/30729.

---------

Co-authored-by: Giteabot <teabot@gitea.io>
2024-04-30 14:34:40 +02:00
Chongyi Zheng
e80466f734
Resolve lint for unused parameter and unnecessary type arguments (#30750)
Resolve all cases for `unused parameter` and `unnecessary type
arguments`

Related: #30729

---------

Co-authored-by: Giteabot <teabot@gitea.io>
2024-04-29 08:47:56 +00:00
Kemal Zebari
9b7af4340c
Perform Newest sort type correctly when sorting issues (#30644)
Should resolve #30642.

Before this commit, we were treating an empty `?sort=` query parameter
as the correct sorting type (which is to sort issues in descending order
by their created UNIX time). But when we perform `sort=latest`, we did
not include this as a type so we would sort by the most recently updated
when reaching the `default` switch statement block.

This commit fixes this by considering the empty string, "latest", and
just any other string that is not mentioned in the switch statement as
sorting by newest.
2024-04-23 15:10:01 +08:00
silverwind
74f0c84fa4
Enable more revive linter rules (#30608)
Noteable additions:

- `redefines-builtin-id` forbid variable names that shadow go builtins
- `empty-lines` remove unnecessary empty lines that `gofumpt` does not
remove for some reason
- `superfluous-else` eliminate more superfluous `else` branches

Rules are also sorted alphabetically and I cleaned up various parts of
`.golangci.yml`.
2024-04-22 11:48:42 +00:00
wxiaoguang
ca5c895efb
Render embedded code preview by permlink in markdown (#30234)
The permlink in markdown will be rendered as a code preview block, like GitHub

Co-authored-by: silverwind <me@silverwind.io>
2024-04-02 17:48:27 +00:00
Lunny Xiao
3f26fe2fa2
Use db.ListOptions directly instead of Paginator interface to make it easier to use and fix performance of /pulls and /issues (#29990)
This PR uses `db.ListOptions` instead of `Paginor` to make the code
simpler.
And it also fixed the performance problem when viewing /pulls or
/issues. Before the counting in fact will also do the search.

---------

Co-authored-by: Jason Song <i@wolfogre.com>
Co-authored-by: silverwind <me@silverwind.io>
2024-03-24 18:51:08 +00:00
wxiaoguang
4734d43e14
Support repo code search without setting up an indexer (#29998)
By using git's ability, end users (especially small instance users) do
not need to enable the indexer, they could also benefit from the code
searching feature.

Fix #29996


![image](https://github.com/go-gitea/gitea/assets/2114189/11b7e458-88a4-480d-b4d7-72ee59406dd1)


![image](https://github.com/go-gitea/gitea/assets/2114189/0fe777d5-c95c-4288-a818-0427680805b6)

---------

Co-authored-by: silverwind <me@silverwind.io>
2024-03-24 17:05:00 +01:00
6543
b9c57fb78e
Determine fuzziness of bleve indexer by keyword length (#29706)
also bleve did match on fuzzy search and the other way around. this also fix that bug.
2024-03-23 16:45:13 +01:00
Lunny Xiao
f8ab9dafb7
Use db.ListOptionsAll instead of db.ListOptions{ListAll: true} (#29995) 2024-03-22 13:53:52 +01:00
6543
c6e5ec51bd
Meilisearch double quote on "match" query (#29740)
make `nonFuzzyWorkaround` unessesary

cc @Kerollmops
2024-03-16 13:19:41 +00:00
6543
1262ff6734
Refactor code_indexer to use an SearchOptions struct for PerformSearch (#29724)
similar to how it's already done for the issue_indexer


---
*Sponsored by Kithara Software GmbH*
2024-03-16 10:32:45 +00:00
6543
7fd0a5b276
Refactor to use optional.Option for issue index search option (#29739)
Signed-off-by: 6543 <6543@obermui.de>
2024-03-13 08:25:53 +00:00
Lunny Xiao
3c6fc25a77
Use repo object format name instead of detecting from git repository (#29702)
It's unnecessary to detect the repository object format from git
repository. Just use the repository's object format name.
2024-03-10 22:30:36 +01:00
6543
7fdc048153
Patch in exact search for meilisearch (#29671)
meilisearch does not have an search option to contorl fuzzynes per query
right now:
 - https://github.com/meilisearch/meilisearch/issues/1192
 - https://github.com/orgs/meilisearch/discussions/377
 - https://github.com/meilisearch/meilisearch/discussions/1096

so we have to create a workaround by post-filter the search result in
gitea until this is addressed.

For future works I added an option in backend only atm, to enable
fuzzynes for issue indexer too.
And also refactored the code so the fuzzy option is equal in logic to
code indexer


---
*Sponsored by Kithara Software GmbH*
2024-03-09 01:39:27 +00:00
yp05327
a2b0fb1a64
Fix wrong line number in code search result (#29260)
Fix #29136

Before: The result is a table and all line numbers are all in one row.

After: Use a separate table column for the line numbers.

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-03-06 07:24:43 +00:00