Commit graph

9 commits

Author SHA1 Message Date
Earl Warren 72c1f20715
[DB] run all Forgejo migrations in integration tests
The tests at tests/integration/migration-test/migration_test.go will
not run any Forgejo migration when using the gitea-*.sql.gz files
because they do not contain a ForgejoVersion row which is interpreted
as a new Forgejo installation for which there is no need for migration.

Create a situation by which the ForgejoVersion table exists and has a
version of 0 in tests/integration/migration-test/forgejo-v1.19.0.*.sql.gz
thus ensuring all Forgejo migrations are run.

The forgejo*.sql.gz files do not have any Gitea related records, which
will be interpreted by the Gitea migrations as a new installation that
does not need any migration. As a consequence the migration tests run
when using forgejo-v1.19.0.*.sql.gz are exclusively about Forgejo
migrations.

(cherry picked from commit ec8003859c920ac05a071ad9b1d9d8af5a694ac0)
(cherry picked from commit 697570ff390656930d41bcdc4194aacc5c7a54f4)
(cherry picked from commit f041aec17265ca34943f961d9640e60c96c0f24d)
(cherry picked from commit 60463e3bab9dfa6aae7f379fc77309ab52b1d250)
(cherry picked from commit b2fc2a7c1338ad2a9bd0a6887443e604dccda53f)
(cherry picked from commit fb2759b6afc213d5e2db8f6e523391927c611b23)
(cherry picked from commit 37cfc3b2275afd523f53e149781d69a2914fd83b)
(cherry picked from commit 832607500a576a40ee1e35897e58ef319dba9788)
(cherry picked from commit 143d4007b1b60ad1880b64ce1d4998e77854a05a)
(cherry picked from commit a17e803fbfb7908729c2d92fd58360806cce6fbe)
(cherry picked from commit 72ffd49bc3c218acf8c87f7464f2c028860adba8)
(cherry picked from commit 9b92a5fd725b34d7ed18b5f1ebc78e1a45de07eb)
(cherry picked from commit 0a334d0a9b02617360aaa76d55dbb0f6fd5547f4)
(cherry picked from commit 3add683c9411018f735c0a19827354870310b9c8)
(cherry picked from commit 8ed3f3f86b0ddab8fc6adc0557b3f5d5e3b8ed4a)
(cherry picked from commit 97dc65852b215d85c8608033bd4180d91aef1a54)
(cherry picked from commit e0503cbfead8e5f7f48d0eb0f32bd6881437bb9b)
(cherry picked from commit f25e6a196b32dcb500fa7df65f21124e5e6a10c4)
(cherry picked from commit cca19c7a4e35982c5fec4de4fcb53690f22d1212)
(cherry picked from commit 2c89d7f18c9493e720e0d06c4b2cd072445eff47)
(cherry picked from commit 4eb8695c40e5d1c64113ce19bb3deae1eafe64c9)
(cherry picked from commit e4c42a68ffccb83fcd7379baa402daae15d6f57b)
(cherry picked from commit f76d5e0a820fbce8c7be968d8318a5a543e43e85)
(cherry picked from commit c6e5a2962cae3eee0b39469bcf39396ee6efddcd)
(cherry picked from commit f9927a2253263110b60c4a49771bf72f660b3ad5)
2024-02-05 14:44:33 +01:00
silverwind 60e4a98ab0
Preserve BOM in web editor ()
The `ToUTF8*` functions were stripping BOM, while BOM is actually valid
in UTF8, so the stripping must be optional depending on use case. This
does:

- Add a options struct to all `ToUTF8*` functions, that by default will
strip BOM to preserve existing behaviour
- Remove `ToUTF8` function, it was dead code
- Rename `ToUTF8WithErr` to `ToUTF8`
- Preserve BOM in Monaco Editor
- Remove a unnecessary newline in the textarea value. Browsers did
ignore it, it seems but it's better not to rely on this behaviour.

Fixes: https://github.com/go-gitea/gitea/issues/28743
Related: https://github.com/go-gitea/gitea/issues/6716 which seems to
have once introduced a mechanism that strips and re-adds the BOM, but
from what I can tell, this mechanism was removed at some point after
that PR.
2024-01-27 18:02:51 +00:00
wxiaoguang 4647660776
Rewrite logger system ()
## ⚠️ Breaking

The `log.<mode>.<logger>` style config has been dropped. If you used it,
please check the new config manual & app.example.ini to make your
instance output logs as expected.

Although many legacy options still work, it's encouraged to upgrade to
the new options.

The SMTP logger is deleted because SMTP is not suitable to collect logs.

If you have manually configured Gitea log options, please confirm the
logger system works as expected after upgrading.

## Description

Close  and maybe more log-related issues, resolve some related
FIXMEs in old code (which seems unfixable before)

Just like rewriting queue  : make code maintainable, clear legacy
bugs, and add the ability to support more writers (eg: JSON, structured
log)

There is a new document (with examples): `logging-config.en-us.md`

This PR is safer than the queue rewriting, because it's just for
logging, it won't break other logic.

## The old problems

The logging system is quite old and difficult to maintain:
* Unclear concepts: Logger, NamedLogger, MultiChannelledLogger,
SubLogger, EventLogger, WriterLogger etc
* Some code is diffuclt to konw whether it is right:
`log.DelNamedLogger("console")` vs `log.DelNamedLogger(log.DEFAULT)` vs
`log.DelLogger("console")`
* The old system heavily depends on ini config system, it's difficult to
create new logger for different purpose, and it's very fragile.
* The "color" trick is difficult to use and read, many colors are
unnecessary, and in the future structured log could help
* It's difficult to add other log formats, eg: JSON format
* The log outputer doesn't have full control of its goroutine, it's
difficult to make outputer have advanced behaviors
* The logs could be lost in some cases: eg: no Fatal error when using
CLI.
* Config options are passed by JSON, which is quite fragile.
* INI package makes the KEY in `[log]` section visible in `[log.sub1]`
and `[log.sub1.subA]`, this behavior is quite fragile and would cause
more unclear problems, and there is no strong requirement to support
`log.<mode>.<logger>` syntax.


## The new design

See `logger.go` for documents.


## Screenshot

<details>


![image](https://github.com/go-gitea/gitea/assets/2114189/4462d713-ba39-41f5-bb08-de912e67e1ff)


![image](https://github.com/go-gitea/gitea/assets/2114189/b188035e-f691-428b-8b2d-ff7b2199b2f9)


![image](https://github.com/go-gitea/gitea/assets/2114189/132e9745-1c3b-4e00-9e0d-15eaea495dee)

</details>

## TODO

* [x] add some new tests
* [x] fix some tests
* [x] test some sub-commands (manually ....)

---------

Co-authored-by: Jason Song <i@wolfogre.com>
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: Giteabot <teabot@gitea.io>
2023-05-21 22:35:11 +00:00
Lunny Xiao 377a0a20f0
Merge setting.InitXXX into one function with options ()
This PR will merge 3 Init functions on setting packages as 1 and
introduce an options struct.
2023-05-04 11:55:35 +08:00
Jason Song c84238800b
Refactor setting.Database.UseXXX to methods ()
Replace .

Refactor `setting.Database.UseMySQL` to
`setting.Database.Type.IsMySQL()`.

To avoid mismatching between `Type` and `UseXXX`.

This refactor can fix the bug mentioned in , so it should be
backported.
2023-03-07 18:51:06 +08:00
Lunny Xiao c53ad052d8
Refactor the setting to make unit test easier ()
Some bugs caused by less unit tests in fundamental packages. This PR
refactor `setting` package so that create a unit test will be easier
than before.

- All `LoadFromXXX` files has been splited as two functions, one is
`InitProviderFromXXX` and `LoadCommonSettings`. The first functions will
only include the code to create or new a ini file. The second function
will load common settings.
- It also renames all functions in setting from `newXXXService` to
`loadXXXSetting` or `loadXXXFrom` to make the function name less
confusing.
- Move `XORMLog` to `SQLLog` because it's a better name for that.

Maybe we should finally move these `loadXXXSetting` into the `XXXInit`
function? Any idea?

---------

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: delvh <dev.lh@web.de>
2023-02-20 00:12:01 +08:00
flynnnnnnnnnn e81ccc406b
Implement FSFE REUSE for golang files ()
Change all license headers to comply with REUSE specification.

Fix 

Co-authored-by: flynnnnnnnnnn <flynnnnnnnnnn@github>
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
2022-11-27 18:20:29 +00:00
Lunny Xiao e72acd5e5b
Split migrations folder ()
There are too many files in `models/migrations` folder so that I split
them into sub folders.
2022-11-02 16:54:36 +08:00
Kyle D c8ded77680
Kd/ci playwright go test ()
* Add initial playwright config

* Simplify Makefile

* Simplify Makefile

* Use correct config files

* Update playwright settings

* Fix package-lock file

* Don't use test logger for e2e tests

* fix frontend lint

* Allow passing TEST_LOGGER variable

* Init postgres database

* use standard gitea env variables

* Update playwright

* update drone

* Move empty env var to commands

* Cleanup

* Move integrations to subfolder

* tests integrations to tests integraton

* Run e2e tests with go test

* Fix linting

* install CI deps

* Add files to ESlint

* Fix drone typo

* Don't log to console in CI

* Use go test http server

* Add build step before tests

* Move shared init function to common package

* fix drone

* Clean up tests

* Fix linting

* Better mocking for page + version string

* Cleanup test generation

* Remove dependency on gitea binary

* Fix linting

* add initial support for running specific tests

* Add ACCEPT_VISUAL variable

* don't require git-lfs

* Add initial documentation

* Review feedback

* Add logged in session test

* Attempt fixing drone race

* Cleanup and bump version

* Bump deps

* Review feedback

* simplify installation

* Fix ci

* Update install docs
2022-09-02 15:18:23 -04:00