Commit graph

236 commits

Author SHA1 Message Date
zeripath 27b351aba5
Make LDAP be able to skip local 2FA (#16954)
This PR extends #16594 to allow LDAP to be able to be set to skip local 2FA too. The technique used here would be extensible to PAM and SMTP sources.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-09-17 12:43:47 +01:00
6543 77f604a928
Add skip and limit to git.GetTags (#16897)
* Make GetTags() api similar to GetBranches()
* Use it for Tag/Release page
2021-09-10 19:30:37 +02:00
6543 2289580bb7
[API] generalize list header (#16551)
* Add info about list endpoints to CONTRIBUTING.md

* Let all list endpoints return X-Total-Count header 

* Add TODOs for GetCombinedCommitStatusByRef

* Fix models/issue_stopwatch.go

* Rrefactor models.ListDeployKeys

* Introduce helper func and use them for SetLinkHeader related func
2021-08-12 14:43:08 +02:00
6543 c4d70a0325
Rename ctx.Form() to ctx.FormString() and move code into own file (#16571)
Followup from #16562 prepare for #16567

* Rename ctx.Form() to ctx.FormString()
* Reimplement FormX func to need less code and cpu cycles
* Move code into own file
2021-08-11 02:31:13 +02:00
zeripath afd88a2418
Allow setting X-FRAME-OPTIONS (#16643)
* Allow setting X-FRAME-OPTIONS

This PR provides a mechanism to set the X-FRAME-OPTIONS header.

Fix #7951

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Update docs/content/doc/advanced/config-cheat-sheet.en-us.md

Co-authored-by: John Olheiser <john.olheiser@gmail.com>

Co-authored-by: John Olheiser <john.olheiser@gmail.com>
2021-08-06 16:47:10 -04:00
zeripath 7c4172ef71
Pass down SignedUserName down to AccessLogger context (#16605)
* Pass down SignedUserName down to AccessLogger context

Unfortunately when the AccessLogger was moved back before the contexters the
SignedUserName reporting was lost. This is due to Request.WithContext leading to a
shallow copy of the Request and the modules/context/Context being within that request.

This PR adds a new context variable of a string pointer which is set and handled
in the contexters.

Fix #16600

Signed-off-by: Andrew Thornton <art27@cantab.net>

* handle nil ptr issue

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-08-04 13:26:30 -04:00
Lunny Xiao 33e0b38287
Rename context.Query to context.Form (#16562) 2021-07-29 03:42:15 +02:00
Lunny Xiao 9f31f3aa8a
Add an abstract json layout to make it's easier to change json library (#16528)
* Add an abstract json layout to make it's easier to change json library

* Fix import

* Fix import sequence

* Fix blank lines

* Fix blank lines
2021-07-24 18:03:58 +02:00
zeripath 5d2e11eedb
Refactor: Move login out of models (#16199)
`models` does far too much. In particular it handles all `UserSignin`.

It shouldn't be responsible for calling LDAP, SMTP or PAM for signing in.

Therefore we should move this code out of `models`.

This code has to depend on `models` - therefore it belongs in `services`.

There is a package in `services` called `auth` and clearly this functionality belongs in there.

Plan:

- [x] Change `auth.Auth` to `auth.Method` - as they represent methods of authentication.
- [x] Move `models.UserSignIn` into `auth`
- [x] Move `models.ExternalUserLogin`
- [x] Move most of the `LoginVia*` methods to `auth` or subpackages
- [x] Move Resynchronize functionality to `auth`
  - Involved some restructuring of `models/ssh_key.go` to reduce the size of this massive file and simplify its files.
- [x] Move the rest of the LDAP functionality in to the ldap subpackage
- [x] Re-factor the login sources to express an interfaces `auth.Source`?
  - I've done this through some smaller interfaces Authenticator and Synchronizable - which would allow us to extend things in future
- [x] Now LDAP is out of models - need to think about modules/auth/ldap and I think all of that functionality might just be moveable
- [x] Similarly a lot Oauth2 functionality need not be in models too and should be moved to services/auth/source/oauth2
  - [x] modules/auth/oauth2/oauth2.go uses xorm... This is naughty - probably need to move this into models.
  - [x] models/oauth2.go - mostly should be in modules/auth/oauth2 or services/auth/source/oauth2 
- [x] More simplifications of login_source.go may need to be done
- Allow wiring in of notify registration -  *this can now easily be done - but I think we should do it in another PR*  - see #16178 
- More refactors...?
  - OpenID should probably become an auth Method but I think that can be left for another PR
  - Methods should also probably be cleaned up  - again another PR I think.
  - SSPI still needs more refactors.* Rename auth.Auth auth.Method
* Restructure ssh_key.go

- move functions from models/user.go that relate to ssh_key to ssh_key
- split ssh_key.go to try create clearer function domains for allow for
future refactors here.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-07-24 11:16:34 +01:00
zeripath d06f9ce274
Redirect on bad CSRF instead of presenting bad page (#14937)
The current CSRF handler is a bit harsh with bad CSRF tokens on webpages
I think we can be a little kinder and redirect to base page with a flash error

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-07-08 15:57:24 +02:00
luzpaz e0296b6a6d
Fix various documentation, user-facing, and source comment typos (#16367)
* Fix various doc, user-facing, and source comment typos

Found via `codespell -q 3 -S ./options/locale,./vendor -L ba,pullrequest,pullrequests,readby`
2021-07-08 13:38:13 +02:00
Lunny Xiao b223d36195
Rework repository archive (#14723)
* Use storage to store archive files

* Fix backend lint

* Add archiver table on database

* Finish archive download

* Fix test

* Add database migrations

* Add status for archiver

* Fix lint

* Add queue

* Add doctor to check and delete old archives

* Improve archive queue

* Fix tests

* improve archive storage

* Delete repo archives

* Add missing fixture

* fix fixture

* Fix fixture

* Fix test

* Fix archiver cleaning

* Fix bug

* Add docs for repository archive storage

* remove repo-archive configuration

* Fix test

* Fix test

* Fix lint

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-06-23 17:12:38 -04:00
6543 6ad5d0a306
[API] ListReleases add filter for draft and pre-releases (#16175)
* invent ctx.QueryOptionalBool

* [API] ListReleases add draft and pre-release filter

* Add X-Total-Count header

* Add a release to fixtures

* Add TEST for API ListReleases
2021-06-17 10:58:10 +02:00
KN4CK3R 440039c0cc
Add push to remote mirror repository (#15157)
* Added push mirror model.

* Integrated push mirror into queue.

* Moved methods into own file.

* Added basic implementation.

* Mirror wiki too.

* Removed duplicated method.

* Get url for different remotes.

* Added migration.

* Unified remote url access.

* Add/Remove push mirror remotes.

* Prevent hangs with missing credentials.

* Moved code between files.

* Changed sanitizer interface.

* Added push mirror backend methods.

* Only update the mirror remote.

* Limit refs on push.

* Added UI part.

* Added missing table.

* Delete mirror if repository gets removed.

* Changed signature. Handle object errors.

* Added upload method.

* Added "upload" unit tests.

* Added transfer adapter unit tests.

* Send correct headers.

* Added pushing of LFS objects.

* Added more logging.

* Simpler body handling.

* Process files in batches to reduce HTTP calls.

* Added created timestamp.

* Fixed invalid column name.

* Changed name to prevent xorm auto setting.

* Remove table header im empty.

* Strip exit code from error message.

* Added docs page about mirroring.

* Fixed date.

* Fixed merge errors.

* Moved test to integrations.

* Added push mirror test.

* Added test.
2021-06-14 19:20:43 +02:00
Lunny Xiao fb3ffeb18d
Add sso.Group, context.Auth, context.APIAuth to allow auth special routes (#16086)
* Add sso.Group, context.Auth, context.APIAuth to allow auth special routes

* Remove unnecessary check

* Rename sso -> auth

* remove unused method of Auth interface
2021-06-09 19:53:16 +02:00
zeripath 3183a465d7
Make modules/context.Context a context.Context (#16031)
* Make modules/context.Context a context.Context

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Simplify context calls

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Set the base context for requests to the HammerContext

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-05-31 02:18:11 -04:00
zeripath 17c5c654a5
Prevent double-login for Git HTTP and LFS and simplify login (#15303)
* Prevent double-login for Git HTTP and LFS and simplify login

There are a number of inconsistencies with our current methods for
logging in for git and lfs. The first is that there is a double login
process. This is particularly evident in 1.13 where there are no less
than 4 hash checks for basic authentication due to the previous
IsPasswordSet behaviour.

This duplicated code had individual inconsistencies that were not
helpful and caused confusion.

This PR does the following:

* Remove the specific login code from the git and lfs handlers except
for the lfs special bearer token
* Simplify the meaning of DisableBasicAuthentication to allow Token and
Oauth2 sign-in.
* The removal of the specific code from git and lfs means that these
both now have the same login semantics and can - if not
DisableBasicAuthentication - login from external services. Further it
allows Oauth2 token authentication as per our standard mechanisms.
* The change in the recovery handler prevents the service from
re-attempting to login - primarily because this could easily cause a
further panic and it is wasteful.

* add test

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Andrew Thornton <art27@cantab.net>
2021-05-15 17:32:09 +02:00
zeripath 2d87a84709
Stop calling WriteHeader in Write (#15862)
Fixes http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)

* Looking again we don't need this writeHeader as all of our downstream
implementations will always do it for us

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
2021-05-14 11:05:50 +03:00
silverwind 1e6fa57acb
Use single shared random string generation function (#15741)
* Use single shared random string generation function

- Replace 3 functions that do the same with 1 shared one
- Use crypto/rand over math/rand for a stronger RNG
- Output only alphanumerical for URL compatibilty

Fixes: #15536

* use const string method

* Update modules/avatar/avatar.go

Co-authored-by: a1012112796 <1012112796@qq.com>

Co-authored-by: a1012112796 <1012112796@qq.com>
2021-05-10 07:45:17 +01:00
zeripath 270aab429e
On open repository open common cat file batch and batch-check (#15667)
Use common git cat-file --batch and git cat-file --batch-check to
significantly reduce calls to git.
    
Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-05-10 03:27:03 +02:00
a1012112796 3088866531
fix some ui bug about draft release (#15137)
* fix some ui bug about draft release

- should not show draft release in tag list because
  it will't create real tag
- still show draft release without tag and commit message
  for draft release instead of 404 error
- remove tag load for attachement links because it's useless

Signed-off-by: a1012112796 <1012112796@qq.com>

* add test code

* fix test

That's because has added a new release in relaese test database.

* fix dropdown link for draft release
2021-05-06 05:12:50 +02:00
zeripath eedc0c8324
Defer closing the gitrepo until the end of the wrapped context functions (#15653)
There was a mistake in #15372 where deferral of gitrepo close occurs before it should.

This PR fixes this.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-05-06 01:30:25 +02:00
zeripath 47fd156936
Use route rather than use thus reducing the number of stack frames (#15301)
Since the move to Chi the number of stack frames has proliferated somewhat catastrophically and we're up to 96 frames with multiple tests of the url outside of a trie which is inefficient.

This PR reduces the number of stack frames by 6 through careful use of Route, moves Captcha into its own router so that it only fires on Captcha routes, similarly for avatars and repo-avatars.

The robots.txt, / and apple-touch-icon.png are moved out of requiring Contexter.

It moves access logger higher in the stack frame because there is no reason why it can't be higher.

Extract from #15186
Contains #15292
2021-05-04 22:48:31 +01:00
KN4CK3R c80d7f33b6
Unified link creation. (#15619) 2021-04-30 19:25:13 +02:00
Kyle D f44543a1bb
Disable Stars config option (#14653)
* Add config option to disable stars

* Replace "stars" with watched in user profile

* Add documentation
2021-04-15 18:53:57 +02:00
zeripath 8e2a8efd84
Prevent superfluous response.WriteHeader (#15456)
This PR simply checks the status before writing the header.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-04-14 14:57:18 +02:00
zeripath 136a20926c
Turn RepoRef and RepoAssignment back into func(*Context) (#15372)
Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-04-09 20:26:08 -04:00
6543 16dea6cebd
[refactor] replace int with httpStatusCodes (#15282)
* replace "200" (int) with "http.StatusOK" (const)

* ctx.Error & ctx.HTML

* ctx.JSON Part1

* ctx.JSON Part2

* ctx.JSON Part3
2021-04-05 11:30:52 -04:00
a1012112796 43fb4921e3
response simple text message for not html request when 404 (#15229)
* response simple text message for not html request when response 404

Signed-off-by: a1012112796 <1012112796@qq.com>
2021-04-01 17:11:42 +02:00
zeripath 9b261f52f0
Add SameSite setting for cookies (#14900)
Add SameSite setting for cookies and rationalise the cookie setting code. Switches SameSite to Lax by default. 

There is a possible future extension of differentiating which cookies could be set at Strict by default but that is for a future PR.

Fix #5583

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-03-07 08:12:43 +00:00
zeripath f0e15250b9
Migrate to use jsoniter instead of encoding/json (#14841)
* Migrate to use jsoniter

* fix tests

* update gitea.com/go-chi/binding

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
2021-03-01 22:08:10 +01:00
6543 a4148c0f12
Repository transfer has to be confirmed, if user can not create repo for new owner (#14792)
* make repo as "pending transfer" if on transfer start doer has no right to create repo in new destination

* if new pending transfer ocured, create UI & Mail notifications
2021-03-01 01:47:30 +01:00
Lunny Xiao bd9361384a
Fix go get (#14758)
* Fix go get

* Fix default branch

Co-authored-by: 6543 <6543@obermui.de>
2021-02-22 00:51:15 +08:00
zeripath 65c940f66c
Prevent endless loop if templates missing (#14752)
Since the chi upgrade if the templates are missing an endless loop will occur if
status/500.tmpl is missing.

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-02-20 12:26:57 +08:00
6543 0d1444751f
[API] Add pagination to ListBranches (#14524)
* make PaginateUserSlice generic -> PaginateSlice

* Add pagination to ListBranches

* add skip, limit to Repository.GetBranches()

* Move routers/api/v1/utils/utils PaginateSlice -> modules/util/paginate.go

* repo_module.GetBranches paginate

* fix & rename & more logging

* better description

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: a1012112796 <1012112796@qq.com>
2021-02-03 20:06:13 +01:00
Lunny Xiao 5e20fd6dbf
Move middlewares to web/middleware (#14480)
Co-authored-by: 6543 <6543@obermui.de>
2021-01-30 10:55:53 +02:00
Lunny Xiao 0cd87d64ff
Update docs and comments to remove macaron (#14491) 2021-01-29 16:35:30 +01:00
Lunny Xiao 87b098f3b6
Fix json charset bug (#14514) 2021-01-29 08:42:47 -05:00
Lunny Xiao f761c82c94
Fix load time bug (#14508) 2021-01-29 05:33:47 +01:00
6543 a7cfb9f2c3
Fix JSON Header response (#14499) 2021-01-27 15:46:22 -06:00
Lunny Xiao a51cc6dea4
Fix access log (#14475)
Fix #14121, #14478.

The `AccessLog` middleware has to be after `Contexter` or `APIContexter` so that we can get `LoginUserName` if possible.
And also there is a **BREAK** change that it removed internal API access log.
2021-01-27 18:46:35 +01:00
Lunny Xiao 41c0776568
Fix captcha (#14488)
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lauris BH <lauris@nix.lv>
2021-01-27 22:56:54 +08:00
Jui-Nan Lin c10503afec
[Feature] add precise search type for Elastic Search (#12869)
* feat: add type query parameters for specifying precise search

* feat: add select dropdown in search box

Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-01-27 12:00:35 +02:00
Lunny Xiao 6433ba0ec3
Move macaron to chi (#14293)
Use [chi](https://github.com/go-chi/chi) instead of the forked [macaron](https://gitea.com/macaron/macaron). Since macaron and chi have conflicts with session share, this big PR becomes a have-to thing. According my previous idea, we can replace macaron step by step but I'm wrong. :( Below is a list of big changes on this PR.

- [x] Define `context.ResponseWriter` interface with an implementation `context.Response`.
- [x] Use chi instead of macaron, and also a customize `Route` to wrap chi so that the router usage is similar as before.
- [x] Create different routers for `web`, `api`, `internal` and `install` so that the codes will be more clear and no magic .
- [x] Use https://github.com/unrolled/render instead of macaron's internal render
- [x] Use https://github.com/NYTimes/gziphandler instead of https://gitea.com/macaron/gzip
- [x] Use https://gitea.com/go-chi/session which is a modified version of https://gitea.com/macaron/session and removed `nodb` support since it will not be maintained. **BREAK**
- [x] Use https://gitea.com/go-chi/captcha which is a modified version of https://gitea.com/macaron/captcha
- [x] Use https://gitea.com/go-chi/cache which is a modified version of https://gitea.com/macaron/cache
- [x] Use https://gitea.com/go-chi/binding which is a modified version of https://gitea.com/macaron/binding
- [x] Use https://github.com/go-chi/cors instead of https://gitea.com/macaron/cors
- [x] Dropped https://gitea.com/macaron/i18n and make a new one in `code.gitea.io/gitea/modules/translation`
- [x] Move validation form structs from `code.gitea.io/gitea/modules/auth` to `code.gitea.io/gitea/modules/forms` to avoid dependency cycle.
- [x] Removed macaron log service because it's not need any more. **BREAK**
- [x] All form structs have to be get by `web.GetForm(ctx)` in the route function but not as a function parameter on routes definition.
- [x] Move Git HTTP protocol implementation to use routers directly.
- [x] Fix the problem that chi routes don't support trailing slash but macaron did.
- [x] `/api/v1/swagger` now will be redirect to `/api/swagger` but not render directly so that `APIContext` will not create a html render. 

Notices:
- Chi router don't support request with trailing slash
- Integration test `TestUserHeatmap` maybe mysql version related. It's failed on my macOS(mysql 5.7.29 installed via brew) but succeed on CI.

Co-authored-by: 6543 <6543@obermui.de>
2021-01-26 16:36:53 +01:00
Andrew Bezold bc05ddc0eb
Redirect on changed user and org name (#11649)
* Add redirect for user

* Add redirect for orgs

* Add user redirect test

* Appease linter

* Add comment to DeleteUserRedirect function

* Fix locale changes

* Fix GetUserByParams

* Fix orgAssignment

* Remove debug logging

* Add redirect prompt

* Dont Export DeleteUserRedirect & only use it within a session

* Unexport newUserRedirect

* cleanup

* Fix & Dedub API code

* Format Template

* Add Migration & rm dublicat

* Refactor: unexport newRepoRedirect() & rm dedub del exec

* if this fails we'll need to re-rename the user directory

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-01-24 16:23:05 +01:00
6543 4acb499f30
just overload to not get it by mistake again ... (#14440) 2021-01-23 21:33:43 +02:00
Lunny Xiao 135b0e502d
Fix log http status is always zero (#14400)
* Fix log http status is always zero

* Fix lint

Co-authored-by: 6543 <6543@obermui.de>
2021-01-19 20:47:43 -05:00
Lunny Xiao a1c9e8f266
Fix windows build error (#14263)
* fix build

* take flash error message back and fix more windows lint error

* performance optimization

* own step to check lint for windows

Co-authored-by: 6543 <6543@obermui.de>
2021-01-06 09:38:00 +08:00
Lunny Xiao 15a475b7db
Fix recovery middleware to render gitea style page. (#13857)
* Some changes to fix recovery

* Move Recovery to middlewares

* Remove trace code

* Fix lint

* add session middleware and remove dependent on macaron for sso

* Fix panic 500 page rendering

* Fix bugs

* Fix fmt

* Fix vendor

* recover unnecessary change

* Fix lint and addd some comments about the copied codes.

* Use util.StatDir instead of com.StatDir

Co-authored-by: 6543 <6543@obermui.de>
2021-01-05 21:05:40 +08:00
silverwind 3a21f8a986
Inline manifest.json (#14038)
* Inline manifest.json

Improve performance by eliminating this separate request and just inline
this small JSON in HTML directly as a data uri.

Also update previously static app name scripts to use AppName.

I've confirmed this as working via "Add to Homescreen" feature which
offered to save the shortcut under the new app name.

* prerender manifest data on startup

* move to settings

* restore setting.AppStartTime and use it on admin page

* use double quotes because template.URL escapes everything

* fix lint

* move variable to global context variable

* delete template file

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
2020-12-22 19:13:50 +08:00