Commit Graph

2047 Commits

Author SHA1 Message Date
Mateusz Mandera ba7f2a22e6 docs: Add redirects for moved pages about stream archiving.
We're renaming "stream deletion" language to "stream archiving"
and these pages were moved in the process, so we should keep redirects
for them for a while.
2021-04-02 22:08:15 -07:00
Alex Vandiver 07779ea879 middleware: Do not trust X-Forwarded-For; use X-Real-Ip, set from nginx.
The `X-Forwarded-For` header is a list of proxies' IP addresses; each
proxy appends the remote address of the host it received its request
from to the list, as it passes the request down.  A naïve parsing, as
SetRemoteAddrFromForwardedFor did, would thus interpret the first
address in the list as the client's IP.

However, clients can pass in arbitrary `X-Forwarded-For` headers,
which would allow them to spoof their IP address.  `nginx`'s behavior
is to treat the addresses as untrusted unless they match an allowlist
of known proxies.  By setting `real_ip_recursive on`, it also allows
this behavior to be applied repeatedly, moving from right to left down
the `X-Forwarded-For` list, stopping at the right-most that is
untrusted.

Rather than re-implement this logic in Django, pass the first
untrusted value that `nginx` computer down into Django via `X-Real-Ip`
header.  This allows consistent IP addresses in logs between `nginx`
and Django.

Proxied calls into Tornado (which don't use UWSGI) already passed this
header, as Tornado logging respects it.
2021-03-31 14:19:38 -07:00
Sumanth V Rao 7f6fe128f6 hotspots: Add TUTORIAL_ENABLED setting to toggle INTRO_HOTSPOTS.
We add a TUTORIAL_ENABLED setting for self-hosters who want to
disable the tutorial entirely on their system. For this, the
default value (True) is placed in default_settings.py, which
can be overwritten by adding an entry in /etc/zulip/settings.py.
2021-03-30 14:46:42 -07:00
Anders Kaseorg fdefc4275a computed_settings: Remove unused TUTORIAL_ENABLED setting.
It’s unused as of commit 88bec16452
(#6621).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-03-30 14:44:09 -07:00
Mateusz Mandera f329878376 migrations: Subscription.is_user_active denormalization - step one.
This adds the is_user_active with the appropriate code for setting the
value correctly in the future. In the following commit a migration to
backfill the value for existing Subscriptions will be added.

To ensure correct user_profile.is_active handling also in tests, we
replace all direct .is_active mutation with calls to appropriate
functions.
2021-03-30 09:19:03 -07:00
shanukun c95061e9b9 refactor: Make acting_user a mandatory kwarg for do_deactivate_user. 2021-03-29 15:51:45 -07:00
shanukun 8f3ae715c0 refactor: Make acting_user a mandatory kwarg for do_reactivate_user. 2021-03-29 15:51:45 -07:00
PIG208 8dd1164979 static: Make debug_mode for default_page_params follow the setting.
For pages that don't have page_params, the default_page_params now
ensures that debug_mode will correctly follow settings.DEBUG.

This allows blueslip exception popups to work on portico pages for
development environment.

Fixes: #17540.
2021-03-26 09:41:12 -07:00
Riken Shah 1d23d13ef5 urls: Add /flush_caches URL to remove all the server-side cache.
This endpoint is introduced for the puppeteer test, to
clear all the caches after each run.
2021-03-25 12:58:36 -07:00
Strifel 209c89be10 ldap: Add option to limit user access to certain realms.
This adds an option for restricting a ldap user
to only be allowed to login into certain realms.
This is done by configuring an attribute mapping of "org_membership"
to an ldap attribute that will contain the list of subdomains the ldap
user is allowed to access. This is analogous to how it's done in SAML.

Co-authored-by: Mateusz Mandera <mateusz.mandera@zulip.com>
2021-03-18 11:19:59 -07:00
Tushar912 55de66f944 api: Support user_id in get_user_presence_endpoint.
This is part of our general process of replacing emails, which are not
static with time, with user_ids when referring to users in the API.

We still keep the `email` reference option, since it can be useful for
linking third-party applications to Zulip on an intranet that might
have a user's corporate email handy and not want to do the extra round
trip to lookup the user.

The name of the parameter, user_id_or_email, was chosen to to make it
clear that the default/preferred option is user_id.

Fixes #14304.
2021-03-11 20:14:04 -08:00
Nikhil Maske 6b34ba048d docs: Add a note in Incoming email integration docs.
The note states the incoming emails are rate-limited and
its current limits.

Fixes #17435.
2021-03-08 12:23:10 -08:00
Alex Vandiver e53be6d043 email: Set an envelope-from which may be different from the From: field.
The envelope-from is used by the MTA if the destination address is not
deliverable.  Route all such mail to the noreply address.
2021-02-24 17:32:28 -08:00
Tushar912 dfafdda9b3 api: Add REST API endpoint for looking up a user by email address.
Add new rest api endpoint GET users/{email} for looking up a user by
email, which is useful especially for corporate API applications that
might already have a user's email address.

Fixes #14302.
2021-02-15 17:38:33 -08:00
Alex Vandiver 30a8f9f845 prod_settings_template: Make "Miscellaneous settings" the last grouping. 2021-02-15 15:53:05 -08:00
Nikhil Maske 04d3f9310a prod_settings_template: Standardize length of heading hashes.
Adjust "mandatory settings" and "Gitlab OAuth" lengths to match the
length of all of the rest of their same-level headings.
2021-02-15 15:53:05 -08:00
Alex Vandiver 19d06f0dd7 prod_settings_template: Extract more sections from "miscellaneous".
This finds three sets of related settings to extract from the
"Miscellaneous" settings section:
 - Service configuration (PostgreSQL, RabbitMQ, Redis, Memcached)
 - Previews (image, URL, and Twitter)
 - Logging and error reporting
2021-02-15 15:53:01 -08:00
Nikhil Maske 5a1c548e1c prod_settings_template: Move incoming email next to outgoing email. 2021-02-15 14:56:43 -08:00
Anders Kaseorg 6e4c3e41dc python: Normalize quotes with Black.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Anders Kaseorg 11741543da python: Reformat with Black, except quotes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Anders Kaseorg 5028c081cb python: Merge concatenated string literals that Black would uglify.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Anders Kaseorg d8d0e7dfff prod_settings_template: Distinguish text from commented code with ##.
This allows the distinction to survive Black reformatting.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Alex Vandiver a315579140 settings: Move LDAP configuration together, in the same order.
Match the order of the variables between `default_settings.py` and
`settings.py`, and move the defaults into `default_settings.py` so
the section does not require any uncommented lines in `settings.py` if
LDAP is not in use.
2021-02-11 17:23:46 -08:00
Nikhil Maske abff97df39 prod_settings_template: Standardize on no space after the "#" for settings lines. 2021-02-05 17:03:07 -08:00
Nikhil Maske 89b25556dc prod_settings_template: Move LDAP up into authentication, and merge part 1/2. 2021-02-05 17:02:27 -08:00
Nikhil Maske 6ad79b4f85 prod_settings_template: Merge the two "misc" sections into one.
Merge the two "misc" sections into one and place all the service
configurations next to each other. Place the TERMS_OF_SERVICE
and PRIVACY_POLICY at the bottom of the "misc" section.
2021-02-05 17:02:27 -08:00
Vishnu KS 03879eec6d email_log: Mark EmailLogBackEnd.log_email as a staticmethod.
We don't access the self attribute in this method.
2021-01-29 14:51:38 -08:00
Vishnu KS 5870de0fa0 dev_settings: Remove email forwarding related settings.
These are now set in dev-secrets.
2021-01-29 14:51:38 -08:00
Vishnu KS 7d8ade3b2a dev_settings: Allow setting the value of EMAIL_USE_TLS from dev-secrets. 2021-01-29 14:51:38 -08:00
Vishnu KS edac24acf1 email_log: Inherit EmailLogBackEnd from smtp.EmailBackend.
EmailLogBackend used to create a new EmailMessage and copy
only certain values from the original EmailMultiAlternatives
object. This resulted in the loss of information and made
it harder to test PRs like
https://github.com/zulip/zulip/pull/17121.

So instead of creating a new EmailMessage, tweak and send the existing
EmailMultiAlternatives object.
2021-01-29 14:51:38 -08:00
Mateusz Mandera 1432067959 dependencies: Upgrade to Django 3.1.
https://docs.djangoproject.com/en/3.1/releases/3.1/

- django.contrib.postgres.fields.JSONField is deprecated and should be
  replaced with models.JSONField
-  The internals of the implementation in the postgresql backend have
   changed a bit in
   f48f671223
   and thus we need to make an ugly tweak in test_runner.
- app_directories.Loader.get_dirs() now returns a list of PosixPath so
  we need to make a small tweak in TwoFactorLoader for that (PosixPath
  is not iterable)

Fixes #16010.
2021-01-26 10:20:00 -08:00
Alex Vandiver c2526844e9 worker: Remove SignupWorker and friends.
ZULIP_FRIENDS_LIST_ID and MAILCHIMP_API_KEY are not currently used in
production.

This removes the unused 'signups' queue and worker.
2021-01-17 11:16:35 -08:00
Aman Agrawal 62d721e859 docs: Remove HipChat migration guide.
As of Feb 15th 2019, Hipchat Cloud and Stride
have reached End Of Life and are no longer
supported by Atlassian. Since it is almost 2 years
now we can remove the migration guides.
2020-12-23 15:43:13 +05:30
Anders Kaseorg 9359dee75b test_settings: Use TEST_EXTERNAL_HOST to override ‘testserver’ default.
This allows test-backend to work even if the user has EXTERNAL_HOST
set to something else.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-12-17 13:07:59 -08:00
Sourabh Rana 419f163906 nginx: Increase file upload size from 25mb to 80mb. 2020-11-19 00:49:49 -08:00
Vishnu KS 7a6285ede7 email testing: Store the SMTP settings in dev-secrets.
It's super annoying to set this up each time I have to test
email templates in gmail.
2020-10-30 11:50:30 -07:00
Vishnu KS fdea49742c apps: Use GitHub API for generating the web app download link. 2020-10-28 23:04:14 -07:00
ryanreh99 dfa7ce5637 uploads: Support non-AWS S3-compatible server.
Boto3 does not allow setting the endpoint url from
the config file. Thus we create a django setting
variable (`S3_ENDPOINT_URL`) which is passed to
service clients and resources of `boto3.Session`.

We also update the uploads-backend documentation
and remove the config environment variable as now
AWS supports the SIGv4 signature format by default.
And the region name is passed as a parameter instead
of creating a config file for just this value.

Fixes #16246.
2020-10-28 21:59:07 -07:00
Alex Vandiver 97745688ca docs: Link to the new doc home of the email gateway. 2020-10-28 12:13:04 -07:00
Alex Vandiver 5eb8064a1a install: Rename postgres options to postgresql. 2020-10-28 11:55:32 -07:00
Alex Vandiver 1f7132f50d docs: Standardize on PostgreSQL, not Postgres. 2020-10-28 11:55:16 -07:00
Anders Kaseorg bb2db31f29 backends: Remove unnecessary deepcopy.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-26 12:31:30 -07:00
Alex Vandiver 1d54630b4e log: Rename email-deliverer.log to match other files. 2020-10-25 14:56:37 -07:00
Anders Kaseorg 72d6ff3c3b docs: Fix more capitalization issues.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-23 11:46:55 -07:00
Mateusz Mandera 48f80fcb0a auth: Expect name in request params in Apple auth.
The name used to be included in the id_token, but this seems to have
been changed by Apple and now it's sent in the `user` request param.

https://github.com/python-social-auth/social-core/pull/483 is the
upstream PR for this - but upstream is currently unmaintained, so we
have to monkey patch.

We also alter the tests to reflect this situation. Tests no longer put
the name in the id_token, but rather in the `user` request param in the
browser flow, just like it happens in reality.

An adaptation has to be made in the native flow - since the name won't
be included by Apple in the id_token anymore, the app, when POSTing
to the /complete/apple/ endpoint,
can (and should for better user experience)
add the `user` param formatted as json of
{"email": "hamlet@zulip.com", "name": {"firstName": "Full", "lastName": "Name"}}
dict. This is also reflected by the change in the
native flow tests.
2020-10-22 12:07:46 -07:00
Tim Abbott 7803a649b3 settings: Clean up Zoom configuration comments. 2020-10-18 15:14:12 -07:00
Vishnu KS b4eafaa022 settings: Typecast FREE_TRIAL_DAYS into integer.
This was missed out in d32362e53e
2020-10-12 17:15:28 -07:00
Anders Kaseorg aabef3d9be python: Catch specific exceptions from orjson.
Followup to #16120.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-11 16:11:41 -07:00
Dinesh 232eb8b7cf auth: Render config error page on configuration error.
We previously used to to redirect to config error page with
a different URL. This commit renders config error in the same
URL where configuration error is encountered. This way when
conifguration error is fixed the user can refresh to continue
normally or go back to login page from the link provided to
choose any other backend auth.

Also moved those URLs to dev_urls.py so that they can be easily
accessed to work on styling etc.

In tests, removed some of the asserts checking status code to be 200
as the function `assert_in_success_response` does that check.
2020-09-25 16:16:17 -07:00
Anders Kaseorg 66ce8903d3 urls: Remove raw string syntax on one line.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-22 15:28:12 -07:00