Commit graph

62 commits

Author SHA1 Message Date
Carlos Felgueiras fc384e494e
Fix validity of the FROM email address not being checked (#29347)
Fixes #27188.
Introduces a check on the installation that tries to parse the FROM
address. If it fails, shows a new error message to the user.

---------

Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
(cherry picked from commit 6f6120dfa8d549d0b866eeb9317054fea831c844)

Conflicts:
	options/locale/locale_en-US.ini
	context
2024-02-26 22:30:26 +01:00
6543 8ff858b94b
Start to migrate from util.OptionalBool to optional.Option[bool] (#29329)
just create transition helper and migrate two structs

(cherry picked from commit 7fbdb60fc1152acc9a040dc04b1b0f5a3475b081)
2024-02-26 22:30:26 +01:00
Earl Warren 094c84ed6d
Merge branch 'rebase-forgejo-dependency' into wip-forgejo 2024-02-05 18:58:23 +01:00
Gusted 464ae81a36
[GITEA] Refactor generation of JWT secret
- Remove non base64-ed version of JWT secret generation. Because all
occurences need the Base64 version.

(cherry picked from commit 6a6b5a31a8e38cb953fcca1c8847ea219234f10c)
(cherry picked from commit 066b8ca6b40a7342352983de35f1ca6683927426)
2024-02-05 16:54:44 +01:00
Gusted d3f9808151
[GITEA] Add password length check on install page
- Resolves #271
- Ensure that the adminstrator password is at least `MIN_PASSWORD_LENGTH`.

(cherry picked from commit 28cb04c3f5040980e716ce66cd5906f324257e02)
(cherry picked from commit 95371ebd92cd005e2d50a4754e60525cf6135b86)
(cherry picked from commit a134288ab6b0291082d913c4e22456b31af58af9)
(cherry picked from commit 4202f052cb32aec71a61dd2afd814035a9d85eea)
(cherry picked from commit 510b7467d3ee0bf346ad1843775affe1df0675ae)
(cherry picked from commit f3a6e1f121e89aaf608fd9890eaf06ed939d1006)
(cherry picked from commit f340508819866f355feec6d01b349fa7df29ace9)
(cherry picked from commit b891bb176d48c3855cc5b6e4573e7a337af9d382)
(cherry picked from commit 1a1bfc38cc7863f5cb3022560cacb2006d08f113)
(cherry picked from commit 083d5aefed10e54814c4438eabcd01973d305502)
(cherry picked from commit 4586096be9b6214058245da3227541866ea4312f)
(cherry picked from commit 039fa20cc8a5b50d5cc37de4503e8a9a80042bcc)
(cherry picked from commit 3ec9cb5f5915cd0bd46ca0d20d0ab798dc7bd135)
(cherry picked from commit 00be0eee3727130966c34a3b95b10f2af06ea2ec)
(cherry picked from commit a1566030025df8cc83d20cbe2b6fb0f87304a1a5)
(cherry picked from commit 4d305e77742c181f68cd24724dfc685723a41b31)
(cherry picked from commit 51e8f21202ea766d69a4b3c26f44c6db07f47844)
(cherry picked from commit 58e354c98e6b361f6d651ffdca3d5cb459adbf2f)
(cherry picked from commit 20405564f56775ba0f29a54c9a6eca8136d8ac99)
(cherry picked from commit 1d7f49568319cfa49e9c8338f2375432f4917739)
(cherry picked from commit d457b9c9111c04ffcd26ff859e2ad804697c2621)
(cherry picked from commit 72b54bc4cce030540310e50acc41ea789a1e5221)
(cherry picked from commit d7ce723e350d21ef42eba7b7013543e2ba6e0e17)
(cherry picked from commit ce5f863d5d3eff77b9736db453f0f9a65241c9bb)
(cherry picked from commit 324b9318acbf5e12be922ee7f8fc0f0fece1743a)
(cherry picked from commit fff11fc535c1a1122914170363bfc23aeb52e02c)
(cherry picked from commit d3fa04aa699883df9b227382190f57726c591cb8)
(cherry picked from commit d3b24691f389d863be834ccc8b2c8910b1614f30)
(cherry picked from commit 736dfab3ae943fb1b87a5468248c5d80887a5e7c)
(cherry picked from commit 8be95ef7f41c9e1d343a89cbfe67bdccc01df1f8)
(cherry picked from commit 0ce04d93a858a61d322750906629ce7da0e22116)
2024-02-05 16:05:50 +01:00
Caesar Schinas 95c371d1bb
[BRANDING] Rebrand default config settings for new installs (#140)
Replaces `Gitea` with `Forgejo` in the default config settings for new installs.

This will not affect existing installs.

Co-authored-by: Caesar Schinas <caesar@caesarschinas.com>
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/140
Co-authored-by: Caesar Schinas <caesar@noreply.codeberg.org>
Co-committed-by: Caesar Schinas <caesar@noreply.codeberg.org>
(cherry picked from commit ca1319aa16128516e50dabdc8e8cadc23eb71d2f)
(cherry picked from commit 52a4d238a0b5bbea28b28e55e35f28c51ecbb2de)
(cherry picked from commit f63536538cfe4b010ebb5a6323f4b5e5b6ec7232)

Conflicts:
	web_src/js/features/install.js
(cherry picked from commit 861cc434e129f3fbd932ee36067c560e754dab9a)
(cherry picked from commit 0e6ea60c802d6cfd95dea4aad5df96bb6e4bc7a5)
(cherry picked from commit 0cbc0ec15d9d952b0ecfb68a36bf58cbae0c43dd)
(cherry picked from commit 3cc19b0ae214ae163f23efa52ab0aa53eb7c935b)
(cherry picked from commit 50fcb885fe2f87a27e6ff778cdd0c7cd97bbe9e9)
(cherry picked from commit f6039d4df481fc58b8db3e863158665d24cac847)
(cherry picked from commit 5ae5c6ba2deefe829f768900f5e1bbcbe9389e15)
(cherry picked from commit f0b565e0ed7fe52f0264e92c49736b487b9bff8a)
(cherry picked from commit adbd4d2015e8e2c4789849c84cfa4032acd40b7f)
(cherry picked from commit d26c540ffdbfb7ec83412635576ab39653d1b531)
(cherry picked from commit 6df6781b42ea126d029ba9e85485dbcb9bf6601d)
(cherry picked from commit b6fb56e1c407195bedfe09f91ecb6537024d5189)
(cherry picked from commit bb4f98a0ca7515aa6c44529df0573195f779a643)
(cherry picked from commit 6779229f2732f6791aba3bd1ba51a338a88ca1f3)
(cherry picked from commit c216c85aee552aa15e9d6316002ee9e170de64d7)
(cherry picked from commit dff780bced1dc78ddd7fa4952766969ee528c538)
(cherry picked from commit 4e036aa3b6d1bc5f2041c3b30ec289cb082fe824)
(cherry picked from commit 8b3bc3e8a64edaba64cb759ea31248eea7307937)
(cherry picked from commit 1e4d8523321efaf6c5febdd77ba5150effe5c1ac)
(cherry picked from commit 07a15d18447bb03bb04001f1f65305670d1467ef)
(cherry picked from commit fb44b3e10d685dd180f37678a3e2a64ee641f2b2)
(cherry picked from commit b212d833190ce59230b3fc288c1aac5106cf33a0)
(cherry picked from commit 5754971be5c37e97f3165878af3760117e40af01)
(cherry picked from commit 0c43b4e82cab028fae1b709c549251d63a3ffc04)

Conflicts:
	routers/install/install.go
	https://codeberg.org/forgejo/forgejo/pulls/1351
(cherry picked from commit 2e22a7208a0f149afc3794425c7b5b2b0181a939)
(cherry picked from commit 676b0a8a48f99d751b677b9eca5b57238b5003a5)
(cherry picked from commit bc4a8bf9bcb75e92b7802c477b1bb14a047344a6)
(cherry picked from commit 5e09a4e174cc6d62b08fa718d0309275effadbd8)
(cherry picked from commit 712c52a32a3b6ce9aafaefaf9e63729522d1e0c5)
(cherry picked from commit ba3d93cc4a5dca5d0daef647fab557e0136925e2)
(cherry picked from commit a5a0396abc512834ca7b0b247277e4d753a87659)
(cherry picked from commit 09b205f30b17b8f22688f39b5f98d06b44745a4b)
(cherry picked from commit 52b7729e86aade4faa9577abb53401a4040131f0)
(cherry picked from commit bfe78735e84312bccb8adb402041830da93ba646)
(cherry picked from commit 915c4f7a7b1e20ba4299a8631c873c7aed288992)
(cherry picked from commit 3b1af856bbc604511bdd42eda65dbf0808a12abc)
(cherry picked from commit ffe2c0a3535fa510ba416d12324d00406678075e)
(cherry picked from commit fd5cc72128272572a84b011f67a05999e60e931a)
(cherry picked from commit 95d3555e47cdca47e603692e669ebbdc024dc6d3)
(cherry picked from commit b9b853f4b0a9a0587a1b2fce7c63bb315aba0af2)
(cherry picked from commit 967453eb6f5776d65883797dd9ad24b57eecbbd3)
(cherry picked from commit 3298f502dda8570884494e557df5800ba0f2150f)
(cherry picked from commit 3481d1f816fe87db000cde4ef1e6d952c520efcf)
(cherry picked from commit 7508c24a0c6d44f98ff1ccb96285309e640b16bb)
(cherry picked from commit 54e97f7c35bbb1800defdfdf99934f22505c0c4d)
(cherry picked from commit 492876992f59c597882c69fef6880d7981e33046)
2024-02-05 16:02:13 +01:00
Gusted c6a53c3172
[SECURITY] Rework long-term authentication
- This is a 'front-port' of the already existing patch on v1.21 and
v1.20, but applied on top of what Gitea has done to rework the LTA
mechanism. Forgejo will stick with the reworked mechanism by the Forgejo
Security team for the time being. The removal of legacy code (AES-GCM) has been
left out.
- The current architecture is inherently insecure, because you can
construct the 'secret' cookie value with values that are available in
the database. Thus provides zero protection when a database is
dumped/leaked.
- This patch implements a new architecture that's inspired from: [Paragonie Initiative](https://paragonie.com/blog/2015/04/secure-authentication-php-with-long-term-persistence#secure-remember-me-cookies).
- Integration testing is added to ensure the new mechanism works.
- Removes a setting, because it's not used anymore.

(cherry picked from commit e3d6622a63da9c33eed1e3d102cf28a92ff653d6)
(cherry picked from commit fef1a6dac5e25579e42d40209c4cfc06879948b9)
(cherry picked from commit b0c5165145fa52f2f7bbec1f50b308bdf1d20ef3)
(cherry picked from commit 7ad51b9f8d0647eecacd258f6ee26155da3872e1)
(cherry picked from commit 64f053f3834e764112cde26bb0d16c5e88d6b2af)
(cherry picked from commit f5e78e4c204ce50b800645d614218b6b6096eecb)

Conflicts:
	services/auth/auth_token_test.go
	https://codeberg.org/forgejo/forgejo/pulls/2069
(cherry picked from commit f69fc23d4bbadf388c7857040ee0774b824e418e)
(cherry picked from commit d955ab3ab02cbb7f1245a8cddec426d64d3ac500)
(cherry picked from commit 9220088f902a25c4690bcabf5a40a8d02e784182)
(cherry picked from commit c73ac636962c41c71814c273510146f0533264ab)
(cherry picked from commit 747a176048ea93085b406429db0e25bb21912eda)

Conflicts:
	models/user/user.go
	routers/web/user/setting/account.go
	https://codeberg.org/forgejo/forgejo/pulls/2295
2024-02-05 15:06:15 +01:00
KN4CK3R c6c829fe3f
Enhanced auth token / remember me (#27606)
Closes #27455

> The mechanism responsible for long-term authentication (the 'remember
me' cookie) uses a weak construction technique. It will hash the user's
hashed password and the rands value; it will then call the secure cookie
code, which will encrypt the user's name with the computed hash. If one
were able to dump the database, they could extract those two values to
rebuild that cookie and impersonate a user. That vulnerability exists
from the date the dump was obtained until a user changed their password.
> 
> To fix this security issue, the cookie could be created and verified
using a different technique such as the one explained at
https://paragonie.com/blog/2015/04/secure-authentication-php-with-long-term-persistence#secure-remember-me-cookies.

The PR removes the now obsolete setting `COOKIE_USERNAME`.
2023-10-14 00:56:41 +00:00
wxiaoguang 9f8d59858a
Refactor system setting (#27000)
This PR reduces the complexity of the system setting system.

It only needs one line to introduce a new option, and the option can be
used anywhere out-of-box.

It is still high-performant (and more performant) because the config
values are cached in the config system.
2023-10-05 09:08:19 +08:00
JakobDev 76659b1114
Reduce usage of db.DefaultContext (#27073)
Part of #27065

This reduces the usage of `db.DefaultContext`. I think I've got enough
files for the first PR. When this is merged, I will continue working on
this.

Considering how many files this PR affect, I hope it won't take to long
to merge, so I don't end up in the merge conflict hell.

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-09-14 17:09:32 +00:00
Lunny Xiao 476b9d1589
Use docs.gitea.com instead of docs.gitea.io (#26739) 2023-08-27 11:59:12 +00:00
wxiaoguang 412e5c0946
Make web context initialize correctly for different cases (#26726)
The web context (modules/context.Context) is quite complex, it's
difficult for the callers to initialize correctly.

This PR introduces a `NewWebContext` function, to make sure the web
context have the same behavior for different cases.
2023-08-25 19:07:42 +08:00
wxiaoguang ed1be4ca68
Handle base64 decoding correctly to avoid panic (#26483)
Fix the panic if the "base64 secret" is too long.
2023-08-14 10:30:16 +00:00
wxiaoguang 6913053223
Start using template context function (#26254)
Before:

* `{{.locale.Tr ...}}`
* `{{$.locale.Tr ...}}`
* `{{$.root.locale.Tr ...}}`
* `{{template "sub" .}}`
* `{{template "sub" (dict "locale" $.locale)}}`
* `{{template "sub" (dict "root" $)}}`
* .....

With context function: only need to `{{ctx.Locale.Tr ...}}`

The "ctx" could be considered as a super-global variable for all
templates including sub-templates.


To avoid potential risks (any bug in the template context function
package), this PR only starts using "ctx" in "head.tmpl" and
"footer.tmpl" and it has a "DataRaceCheck". If there is anything wrong,
the code can be fixed or reverted easily.
2023-08-08 01:22:47 +00:00
wxiaoguang cee352bb38
Show correct SSL Mode on "install page" (#25818) 2023-07-11 18:09:23 -04:00
wxiaoguang fa0b5b14c2
Make "install page" respect environment config (#25648)
Replace #25580

Fix #19453

The problem was: when users set "GITEA__XXX__YYY" , the "install page"
doesn't respect it.

So, to make the result consistent and avoid surprising end users, now
the "install page" also writes the environment variables to the config
file.

And, to make things clear, there are enough messages on the UI to tell
users what will happen.

There are some necessary/related changes to `environment-to-ini.go`:

* The "--clear" flag is removed and it was incorrectly written there.
The "clear" operation should be done if INSTALL_LOCK=true
* The "--prefix" flag is removed because it's never used, never
documented and it only causes inconsistent behavior.


![image](https://github.com/go-gitea/gitea/assets/2114189/12778ee4-3fb5-4664-a73a-41ebbd77cd5b)
2023-07-09 22:43:37 +00:00
wxiaoguang ce46834b93
Remove "CHARSET" config option for MySQL, always use "utf8mb4" (#25413)
In modern days, there is no reason to make users set "charset" anymore.

Close #25378

## ⚠️ BREAKING

The key `[database].CHARSET` was removed completely as every newer
(>10years) MySQL database supports `utf8mb4` already.
There is a (deliberately) undocumented new fallback option if anyone
still needs to use it, but we don't recommend using it as it simply
causes problems.
2023-06-21 10:49:25 +00:00
wxiaoguang 2cdf260f42
Refactor path & config system (#25330)
# The problem

There were many "path tricks":

* By default, Gitea uses its program directory as its work path
* Gitea tries to use the "work path" to guess its "custom path" and
"custom conf (app.ini)"
* Users might want to use other directories as work path
* The non-default work path should be passed to Gitea by GITEA_WORK_DIR
or "--work-path"
* But some Gitea processes are started without these values
    * The "serv" process started by OpenSSH server
    * The CLI sub-commands started by site admin
* The paths are guessed by SetCustomPathAndConf again and again
* The default values of "work path / custom path / custom conf" can be
changed when compiling

# The solution

* Use `InitWorkPathAndCommonConfig` to handle these path tricks, and use
test code to cover its behaviors.
* When Gitea's web server runs, write the WORK_PATH to "app.ini", this
value must be the most correct one, because if this value is not right,
users would find that the web UI doesn't work and then they should be
able to fix it.
* Then all other sub-commands can use the WORK_PATH in app.ini to
initialize their paths.
* By the way, when Gitea starts for git protocol, it shouldn't output
any log, otherwise the git protocol gets broken and client blocks
forever.

The "work path" priority is: WORK_PATH in app.ini > cmd arg --work-path
> env var GITEA_WORK_DIR > builtin default

The "app.ini" searching order is: cmd arg --config > cmd arg "work path
/ custom path" > env var "work path / custom path" > builtin default


## ⚠️ BREAKING

If your instance's "work path / custom path / custom conf" doesn't meet
the requirements (eg: work path must be absolute), Gitea will report a
fatal error and exit. You need to set these values according to the
error log.



----

Close #24818
Close #24222
Close #21606
Close #21498
Close #25107
Close #24981
Maybe close #24503

Replace #23301
Replace #22754

And maybe more
2023-06-21 13:50:26 +08:00
wxiaoguang 1ea6b8fd76
Write absolute AppDataPath to app.ini when installing (#25331)
If the APP_DATA_PATH isn't written into the config when installing, then
its value is uncertain because some Gitea command doesn't run with
correct WorkPath.

This is a quick fix for #25330  and can be backported.
2023-06-18 13:57:43 +00:00
Lunny Xiao d6dd6d641b
Fix all possible setting error related storages and added some tests (#23911)
Follow up #22405

Fix #20703 

This PR rewrites storage configuration read sequences with some breaks
and tests. It becomes more strict than before and also fixed some
inherit problems.

- Move storage's MinioConfig struct into setting, so after the
configuration loading, the values will be stored into the struct but not
still on some section.
- All storages configurations should be stored on one section,
configuration items cannot be overrided by multiple sections. The
prioioty of configuration is `[attachment]` > `[storage.attachments]` |
`[storage.customized]` > `[storage]` > `default`
- For extra override configuration items, currently are `SERVE_DIRECT`,
`MINIO_BASE_PATH`, `MINIO_BUCKET`, which could be configured in another
section. The prioioty of the override configuration is `[attachment]` >
`[storage.attachments]` > `default`.
- Add more tests for storages configurations.
- Update the storage documentations.

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-06-14 11:42:38 +08:00
wxiaoguang eaf551b3c1
Do not overwrite the log mode when installing (#25203)
Fix #24861
2023-06-12 10:52:49 +00:00
wxiaoguang de4a21fcb4
Refactor INI package (first step) (#25024)
The INI package has many bugs and quirks, and in fact it is
unmaintained.

This PR is the first step for the INI package refactoring: 

* Use Gitea's "config_provider" to provide INI access
* Deprecate the INI package by golangci.yml rule
2023-06-02 17:27:30 +08:00
wxiaoguang abcf5a7b5e
Fix install page context, make the install page tests really test (#24858)
Fix #24856

Rename "context.contextKey" to "context.WebContextKey", this context is
for web context only. But the Context itself is not renamed, otherwise
it would cause a lot of changes (if we really want to rename it, there
could be a separate PR).

The old test code doesn't really test, the "install page" gets broken
not only one time, so use new test code to make sure the "install page"
could work.
2023-05-23 09:29:15 +08:00
wxiaoguang 6b33152b7d
Decouple the different contexts from each other (#24786)
Replace #16455

Close #21803

Mixing different Gitea contexts together causes some problems:

1. Unable to respond proper content when error occurs, eg: Web should
respond HTML while API should respond JSON
2. Unclear dependency, eg: it's unclear when Context is used in
APIContext, which fields should be initialized, which methods are
necessary.


To make things clear, this PR introduces a Base context, it only
provides basic Req/Resp/Data features.

This PR mainly moves code. There are still many legacy problems and
TODOs in code, leave unrelated changes to future PRs.
2023-05-21 09:50:53 +08:00
wxiaoguang 5d77691d42
Improve template system and panic recovery (#24461)
Partially for #24457

Major changes:

1. The old `signedUserNameStringPointerKey` is quite hacky, use
`ctx.Data[SignedUser]` instead
2. Move duplicate code from `Contexter` to `CommonTemplateContextData`
3. Remove incorrect copying&pasting code `ctx.Data["Err_Password"] =
true` in API handlers
4. Use one unique `RenderPanicErrorPage` for panic error page rendering
5. Move `stripSlashesMiddleware` to be the first middleware
6. Install global panic recovery handler, it works for both `install`
and `web`
7. Make `500.tmpl` only depend minimal template functions/variables,
avoid triggering new panics

Screenshot:

<details>

![image](https://user-images.githubusercontent.com/2114189/235444895-cecbabb8-e7dc-4360-a31c-b982d11946a7.png)

</details>
2023-05-04 14:36:34 +08:00
wxiaoguang e3750370df
Use globally shared HTMLRender (#24436)
The old `HTMLRender` is not ideal.

1. It shouldn't be initialized multiple times, it consumes a lot of
memory and is slow.
2. It shouldn't depend on short-lived requests, the `WatchLocalChanges`
needs a long-running context.
3. It doesn't make sense to use FuncsMap slice.


HTMLRender was designed to only work for GItea's specialized 400+
templates, so it's good to make it a global shared instance.
2023-04-30 08:22:23 -04:00
wxiaoguang 5b9557aef5
Refactor cookie (#24107)
Close #24062

At the beginning, I just wanted to fix the warning mentioned by #24062

But, the cookie code really doesn't look good to me, so clean up them.

Complete the TODO on `SetCookie`: 

> TODO: Copied from gitea.com/macaron/macaron and should be improved
after macaron removed.
2023-04-13 15:45:33 -04:00
Jason Song c84238800b
Refactor setting.Database.UseXXX to methods (#23354)
Replace #23350.

Refactor `setting.Database.UseMySQL` to
`setting.Database.Type.IsMySQL()`.

To avoid mismatching between `Type` and `UseXXX`.

This refactor can fix the bug mentioned in #23350, so it should be
backported.
2023-03-07 18:51:06 +08:00
wxiaoguang b2359f3df6
Fix various bugs for "install" page (#23194)
## TLDR

* Fix the broken page / broken image problem when click "Install"
* Close #20089
* Fix the Password Hash Algorithm display problem for #22942
* Close #23183
* Close #23184

## Details

### The broken page / broken image problem when click "Install"
(Redirect failed after install gitea #23184)

Before: when click "install", all new requests will fail, because the
server has been restarted. Users just see a broken page with broken
images, sometimes the server is not ready but the user would have been
redirect to "/user/login" page, then the users see a new broken page
(connection refused or something wrong ...)

After: only check InstallLock=true for necessary handlers, and sleep for
a while before restarting the server, then the browser has enough time
to load the "post-install" page. And there is a script to check whether
"/user/login" is ready, the user will only be redirected to the login
page when the server is ready.

### During new instance setup make 'Gitea Base URL' filled from
window.location.origin #20089

If the "app_url" input contains `localhost` (the default value from
config), use current window's location href as the `app_url` (aka
ROOT_URL)

### Fix the Password Hash Algorithm display problem for "Provide the
ability to set password hash algorithm parameters #22942"

Before: the UI shows `pbkdf2$50000$50`

<details>

![image](https://user-images.githubusercontent.com/2114189/221917143-e1e54798-1698-4fee-a18d-00c48081fc39.png)

</details>

After: the UI shows `pbkdf2`

<details>

![image](https://user-images.githubusercontent.com/2114189/221916999-97a15be8-2ebb-4a01-bf93-dac18e354fcc.png)

</details>

### GET data: net::ERR_INVALID_URL #23183

Cause by empty `data:` in `<link rel="manifest"
href="data:{{.ManifestData}}">`

---------

Co-authored-by: Jason Song <i@wolfogre.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2023-03-04 10:12:02 +08:00
Lunny Xiao c53ad052d8
Refactor the setting to make unit test easier (#22405)
Some bugs caused by less unit tests in fundamental packages. This PR
refactor `setting` package so that create a unit test will be easier
than before.

- All `LoadFromXXX` files has been splited as two functions, one is
`InitProviderFromXXX` and `LoadCommonSettings`. The first functions will
only include the code to create or new a ini file. The second function
will load common settings.
- It also renames all functions in setting from `newXXXService` to
`loadXXXSetting` or `loadXXXFrom` to make the function name less
confusing.
- Move `XORMLog` to `SQLLog` because it's a better name for that.

Maybe we should finally move these `loadXXXSetting` into the `XXXInit`
function? Any idea?

---------

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: delvh <dev.lh@web.de>
2023-02-20 00:12:01 +08:00
zeripath 61b89747ed
Provide the ability to set password hash algorithm parameters (#22942)
This PR refactors and improves the password hashing code within gitea
and makes it possible for server administrators to set the password
hashing parameters

In addition it takes the opportunity to adjust the settings for `pbkdf2`
in order to make the hashing a little stronger.

The majority of this work was inspired by PR #14751 and I would like to
thank @boppy for their work on this.

Thanks to @gusted for the suggestion to adjust the `pbkdf2` hashing
parameters.

Close #14751

---------

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-02-19 15:35:20 +08:00
Lunny Xiao bd820aa9c5
Add context cache as a request level cache (#22294)
To avoid duplicated load of the same data in an HTTP request, we can set
a context cache to do that. i.e. Some pages may load a user from a
database with the same id in different areas on the same page. But the
code is hidden in two different deep logic. How should we share the
user? As a result of this PR, now if both entry functions accept
`context.Context` as the first parameter and we just need to refactor
`GetUserByID` to reuse the user from the context cache. Then it will not
be loaded twice on an HTTP request.

But of course, sometimes we would like to reload an object from the
database, that's why `RemoveContextData` is also exposed.

The core context cache is here. It defines a new context
```go
type cacheContext struct {
	ctx  context.Context
	data map[any]map[any]any
        lock sync.RWMutex
}

var cacheContextKey = struct{}{}

func WithCacheContext(ctx context.Context) context.Context {
	return context.WithValue(ctx, cacheContextKey, &cacheContext{
		ctx:  ctx,
		data: make(map[any]map[any]any),
	})
}
```

Then you can use the below 4 methods to read/write/del the data within
the same context.

```go
func GetContextData(ctx context.Context, tp, key any) any
func SetContextData(ctx context.Context, tp, key, value any)
func RemoveContextData(ctx context.Context, tp, key any)
func GetWithContextCache[T any](ctx context.Context, cacheGroupKey string, cacheTargetID any, f func() (T, error)) (T, error)
```

Then let's take a look at how `system.GetString` implement it.

```go
func GetSetting(ctx context.Context, key string) (string, error) {
	return cache.GetWithContextCache(ctx, contextCacheKey, key, func() (string, error) {
		return cache.GetString(genSettingCacheKey(key), func() (string, error) {
			res, err := GetSettingNoCache(ctx, key)
			if err != nil {
				return "", err
			}
			return res.SettingValue, nil
		})
	})
}
```

First, it will check if context data include the setting object with the
key. If not, it will query from the global cache which may be memory or
a Redis cache. If not, it will get the object from the database. In the
end, if the object gets from the global cache or database, it will be
set into the context cache.

An object stored in the context cache will only be destroyed after the
context disappeared.
2023-02-15 21:37:34 +08:00
John Olheiser 32e863439c
fix: update settings table on install (#22326)
Related to #22325 

Signed-off-by: jolheiser <john.olheiser@gmail.com>
2023-01-03 14:33:41 -06:00
flynnnnnnnnnn e81ccc406b
Implement FSFE REUSE for golang files (#21840)
Change all license headers to comply with REUSE specification.

Fix #16132

Co-authored-by: flynnnnnnnnnn <flynnnnnnnnnn@github>
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
2022-11-27 18:20:29 +00:00
wxiaoguang 4d1e2b8334
Fix token generation when using INTERNAL_TOKEN_URI (#21669)
Fix https://github.com/go-gitea/gitea/issues/21666
Caused by https://github.com/go-gitea/gitea/pull/19663

Before: when install, the INTERNAL_TOKEN was always generated and saved.
But the internal token may be already there by INTERNAL_TOKEN_URI

After: INTERNAL_TOKEN_URI file must be non-empty. When install, skip
internal token generation if the token exists.
2022-11-03 20:55:09 +00:00
Gusted c2d2323fc8
Configure update checker on installation page (#21655)
- I recently became aware that this was enabled by-default, I don't
necessary agree with that and this should rather be configured by the
user(this patch does that on the installation page) as it connects to a
homeserver, which I'd prefer to avoid on my instance.


![image](https://user-images.githubusercontent.com/25481501/199260613-a77a1b10-347a-4542-8982-9b9b24dad28c.png)
2022-11-01 19:23:56 +00:00
Lunny Xiao f0182d8840
Fix read system configuration bug when installing (#21489)
Fix https://github.com/go-gitea/gitea/pull/18058#issuecomment-1280944311

Co-authored-by: 6543 <6543@obermui.de>
2022-10-18 23:16:58 +08:00
Lunny Xiao f860a6d2e4
Add system setting table with cache and also add cache supports for user setting (#18058) 2022-10-17 07:29:26 +08:00
zeripath bb0ff77e46
Share HTML template renderers and create a watcher framework (#20218)
The recovery, API, Web and package frameworks all create their own HTML
Renderers. This increases the memory requirements of Gitea
unnecessarily with duplicate templates being kept in memory.

Further the reloading framework in dev mode for these involves locking
and recompiling all of the templates on each load. This will potentially
hide concurrency issues and it is inefficient.

This PR stores the templates renderer in the context and stores this
context in the NormalRoutes, it then creates a fsnotify.Watcher
framework to watch files.

The watching framework is then extended to the mailer templates which
were previously not being reloaded in dev.

Then the locales are simplified to a similar structure.

Fix #20210 
Fix #20211
Fix #20217

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-08-28 10:43:25 +01:00
Clar Fon 036dd8a788
Rework mailer settings (#18982)
* `PROTOCOL`: can be smtp, smtps, smtp+startls, smtp+unix, sendmail, dummy
* `SMTP_ADDR`: domain for SMTP, or path to unix socket
* `SMTP_PORT`: port for SMTP; defaults to 25 for `smtp`, 465 for `smtps`, and 587 for `smtp+startls`
* `ENABLE_HELO`, `HELO_HOSTNAME`: reverse `DISABLE_HELO` to `ENABLE_HELO`; default to false + system hostname
* `FORCE_TRUST_SERVER_CERT`: replace the unclear `SKIP_VERIFY`
* `CLIENT_CERT_FILE`, `CLIENT_KEY_FILE`, `USE_CLIENT_CERT`: clarify client certificates here

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-08-02 13:24:18 +08:00
Gusted d55a0b7238
Refactor i18n to locale (#20153)
* Refactor `i18n` to `locale`

- Currently we're using the `i18n` variable naming for the `locale`
struct. This contains locale's specific information and cannot be used
for general i18n purpose, therefore refactoring it to `locale` makes
more sense.
- Ref: https://github.com/go-gitea/gitea/pull/20096#discussion_r906699200

* Update routers/install/install.go
2022-06-27 15:58:46 -05:00
ttys3 0c759fd4de
feat: add DEFAULT_MERGE_STYLE to repository.pull-request section for repo init (#19751) 2022-06-02 23:45:54 -04:00
Lunny Xiao fd7d83ace6
Move almost all functions' parameter db.Engine to context.Context (#19748)
* Move almost all functions' parameter db.Engine to context.Context
* remove some unnecessary wrap functions
2022-05-20 22:08:52 +08:00
wxiaoguang 04fc4b7e05
Call MultipartForm.RemoveAll when request finishes (#19606) 2022-05-05 16:13:23 +02:00
Vladimir Smagin 0ba3ada866
add smtp password to install page (#17564)
* add smtp password to install page

* Update routers/install/install.go

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-05-02 16:45:23 +08:00
Jimmy Praet 5aebc4f000
Respect DefaultUserIsRestricted system default when creating new user (#19310)
* Apply DefaultUserIsRestricted in CreateUser

* Enforce system defaults in CreateUser

Allow for overwrites with CreateUserOverwriteOptions

* Fix compilation errors

* Add "restricted" option to create user command

* Add "restricted" option to create user admin api

* Respect default setting.Service.RegisterEmailConfirm and setting.Service.RegisterManualConfirm where needed

* Revert "Respect default setting.Service.RegisterEmailConfirm and setting.Service.RegisterManualConfirm where needed"

This reverts commit ee95d3e8dc9e9fff4fa66a5111e4d3930280e033.
2022-04-29 15:38:11 -04:00
wxiaoguang 5b7466053d
Skip frontend ROOT_URL check on installation page, remove unnecessary global var (#19291)
Skip `checkAppUrl` message on installation page because the ROOT_URL is not determined yet
Move global var `supportedDbTypeNames` into `install.Init` as a local var
2022-04-01 16:00:26 +08:00
KN4CK3R 3f280f89e7
Update HTTP status codes to modern codes (#18063)
* 2xx/3xx/4xx/5xx -> http.Status...
* http.StatusFound -> http.StatusTemporaryRedirect
* http.StatusMovedPermanently -> http.StatusPermanentRedirect
2022-03-23 12:54:07 +08:00
wxiaoguang a60e8be8d1
Refactor i18n, use Locale to provide i18n/translation related functions (#18648)
* remove unnecessary web context data fields, and unify the i18n/translation related functions to `Locale`
* in development, show an error if a translation key is missing
* remove the unnecessary loops `for _, lang := range translation.AllLangs()` for every request, which improves the performance slightly
* use `ctx.Locale.Language()` instead of `ctx.Data["Lang"].(string)`
* add more comments about how the Locale/LangType fields are used
2022-02-08 11:02:30 +08:00
Gusted 5e5740af69
Switch to non-deprecation setting (#18358)
* Switch to non-deprecation setting
  (Avoid by-default: "Deprecated fallback `[server]` `LFS_CONTENT_PATH` present. Use `[lfs]` `PATH` instead. This fallback will be removed in v1.18.0")

* Update all references
2022-01-23 20:02:29 +01:00