Commit Graph

2035 Commits

Author SHA1 Message Date
Alex Vandiver e53be6d043 email: Set an envelope-from which may be different from the From: field.
The envelope-from is used by the MTA if the destination address is not
deliverable.  Route all such mail to the noreply address.
2021-02-24 17:32:28 -08:00
Tushar912 dfafdda9b3 api: Add REST API endpoint for looking up a user by email address.
Add new rest api endpoint GET users/{email} for looking up a user by
email, which is useful especially for corporate API applications that
might already have a user's email address.

Fixes #14302.
2021-02-15 17:38:33 -08:00
Alex Vandiver 30a8f9f845 prod_settings_template: Make "Miscellaneous settings" the last grouping. 2021-02-15 15:53:05 -08:00
Nikhil Maske 04d3f9310a prod_settings_template: Standardize length of heading hashes.
Adjust "mandatory settings" and "Gitlab OAuth" lengths to match the
length of all of the rest of their same-level headings.
2021-02-15 15:53:05 -08:00
Alex Vandiver 19d06f0dd7 prod_settings_template: Extract more sections from "miscellaneous".
This finds three sets of related settings to extract from the
"Miscellaneous" settings section:
 - Service configuration (PostgreSQL, RabbitMQ, Redis, Memcached)
 - Previews (image, URL, and Twitter)
 - Logging and error reporting
2021-02-15 15:53:01 -08:00
Nikhil Maske 5a1c548e1c prod_settings_template: Move incoming email next to outgoing email. 2021-02-15 14:56:43 -08:00
Anders Kaseorg 6e4c3e41dc python: Normalize quotes with Black.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Anders Kaseorg 11741543da python: Reformat with Black, except quotes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Anders Kaseorg 5028c081cb python: Merge concatenated string literals that Black would uglify.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Anders Kaseorg d8d0e7dfff prod_settings_template: Distinguish text from commented code with ##.
This allows the distinction to survive Black reformatting.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Alex Vandiver a315579140 settings: Move LDAP configuration together, in the same order.
Match the order of the variables between `default_settings.py` and
`settings.py`, and move the defaults into `default_settings.py` so
the section does not require any uncommented lines in `settings.py` if
LDAP is not in use.
2021-02-11 17:23:46 -08:00
Nikhil Maske abff97df39 prod_settings_template: Standardize on no space after the "#" for settings lines. 2021-02-05 17:03:07 -08:00
Nikhil Maske 89b25556dc prod_settings_template: Move LDAP up into authentication, and merge part 1/2. 2021-02-05 17:02:27 -08:00
Nikhil Maske 6ad79b4f85 prod_settings_template: Merge the two "misc" sections into one.
Merge the two "misc" sections into one and place all the service
configurations next to each other. Place the TERMS_OF_SERVICE
and PRIVACY_POLICY at the bottom of the "misc" section.
2021-02-05 17:02:27 -08:00
Vishnu KS 03879eec6d email_log: Mark EmailLogBackEnd.log_email as a staticmethod.
We don't access the self attribute in this method.
2021-01-29 14:51:38 -08:00
Vishnu KS 5870de0fa0 dev_settings: Remove email forwarding related settings.
These are now set in dev-secrets.
2021-01-29 14:51:38 -08:00
Vishnu KS 7d8ade3b2a dev_settings: Allow setting the value of EMAIL_USE_TLS from dev-secrets. 2021-01-29 14:51:38 -08:00
Vishnu KS edac24acf1 email_log: Inherit EmailLogBackEnd from smtp.EmailBackend.
EmailLogBackend used to create a new EmailMessage and copy
only certain values from the original EmailMultiAlternatives
object. This resulted in the loss of information and made
it harder to test PRs like
https://github.com/zulip/zulip/pull/17121.

So instead of creating a new EmailMessage, tweak and send the existing
EmailMultiAlternatives object.
2021-01-29 14:51:38 -08:00
Mateusz Mandera 1432067959 dependencies: Upgrade to Django 3.1.
https://docs.djangoproject.com/en/3.1/releases/3.1/

- django.contrib.postgres.fields.JSONField is deprecated and should be
  replaced with models.JSONField
-  The internals of the implementation in the postgresql backend have
   changed a bit in
   f48f671223
   and thus we need to make an ugly tweak in test_runner.
- app_directories.Loader.get_dirs() now returns a list of PosixPath so
  we need to make a small tweak in TwoFactorLoader for that (PosixPath
  is not iterable)

Fixes #16010.
2021-01-26 10:20:00 -08:00
Alex Vandiver c2526844e9 worker: Remove SignupWorker and friends.
ZULIP_FRIENDS_LIST_ID and MAILCHIMP_API_KEY are not currently used in
production.

This removes the unused 'signups' queue and worker.
2021-01-17 11:16:35 -08:00
Aman Agrawal 62d721e859 docs: Remove HipChat migration guide.
As of Feb 15th 2019, Hipchat Cloud and Stride
have reached End Of Life and are no longer
supported by Atlassian. Since it is almost 2 years
now we can remove the migration guides.
2020-12-23 15:43:13 +05:30
Anders Kaseorg 9359dee75b test_settings: Use TEST_EXTERNAL_HOST to override ‘testserver’ default.
This allows test-backend to work even if the user has EXTERNAL_HOST
set to something else.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-12-17 13:07:59 -08:00
Sourabh Rana 419f163906 nginx: Increase file upload size from 25mb to 80mb. 2020-11-19 00:49:49 -08:00
Vishnu KS 7a6285ede7 email testing: Store the SMTP settings in dev-secrets.
It's super annoying to set this up each time I have to test
email templates in gmail.
2020-10-30 11:50:30 -07:00
Vishnu KS fdea49742c apps: Use GitHub API for generating the web app download link. 2020-10-28 23:04:14 -07:00
ryanreh99 dfa7ce5637 uploads: Support non-AWS S3-compatible server.
Boto3 does not allow setting the endpoint url from
the config file. Thus we create a django setting
variable (`S3_ENDPOINT_URL`) which is passed to
service clients and resources of `boto3.Session`.

We also update the uploads-backend documentation
and remove the config environment variable as now
AWS supports the SIGv4 signature format by default.
And the region name is passed as a parameter instead
of creating a config file for just this value.

Fixes #16246.
2020-10-28 21:59:07 -07:00
Alex Vandiver 97745688ca docs: Link to the new doc home of the email gateway. 2020-10-28 12:13:04 -07:00
Alex Vandiver 5eb8064a1a install: Rename postgres options to postgresql. 2020-10-28 11:55:32 -07:00
Alex Vandiver 1f7132f50d docs: Standardize on PostgreSQL, not Postgres. 2020-10-28 11:55:16 -07:00
Anders Kaseorg bb2db31f29 backends: Remove unnecessary deepcopy.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-26 12:31:30 -07:00
Alex Vandiver 1d54630b4e log: Rename email-deliverer.log to match other files. 2020-10-25 14:56:37 -07:00
Anders Kaseorg 72d6ff3c3b docs: Fix more capitalization issues.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-23 11:46:55 -07:00
Mateusz Mandera 48f80fcb0a auth: Expect name in request params in Apple auth.
The name used to be included in the id_token, but this seems to have
been changed by Apple and now it's sent in the `user` request param.

https://github.com/python-social-auth/social-core/pull/483 is the
upstream PR for this - but upstream is currently unmaintained, so we
have to monkey patch.

We also alter the tests to reflect this situation. Tests no longer put
the name in the id_token, but rather in the `user` request param in the
browser flow, just like it happens in reality.

An adaptation has to be made in the native flow - since the name won't
be included by Apple in the id_token anymore, the app, when POSTing
to the /complete/apple/ endpoint,
can (and should for better user experience)
add the `user` param formatted as json of
{"email": "hamlet@zulip.com", "name": {"firstName": "Full", "lastName": "Name"}}
dict. This is also reflected by the change in the
native flow tests.
2020-10-22 12:07:46 -07:00
Tim Abbott 7803a649b3 settings: Clean up Zoom configuration comments. 2020-10-18 15:14:12 -07:00
Vishnu KS b4eafaa022 settings: Typecast FREE_TRIAL_DAYS into integer.
This was missed out in d32362e53e
2020-10-12 17:15:28 -07:00
Anders Kaseorg aabef3d9be python: Catch specific exceptions from orjson.
Followup to #16120.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-11 16:11:41 -07:00
Dinesh 232eb8b7cf auth: Render config error page on configuration error.
We previously used to to redirect to config error page with
a different URL. This commit renders config error in the same
URL where configuration error is encountered. This way when
conifguration error is fixed the user can refresh to continue
normally or go back to login page from the link provided to
choose any other backend auth.

Also moved those URLs to dev_urls.py so that they can be easily
accessed to work on styling etc.

In tests, removed some of the asserts checking status code to be 200
as the function `assert_in_success_response` does that check.
2020-09-25 16:16:17 -07:00
Anders Kaseorg 66ce8903d3 urls: Remove raw string syntax on one line.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-22 15:28:12 -07:00
Anders Kaseorg 0228acf0f5 rest: Add rest_path shortcut for path with rest_dispatch.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-22 10:51:00 -07:00
Anders Kaseorg 5297e4a30a urls: Use unqualified imports.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-22 10:51:00 -07:00
Anders Kaseorg e70f2ae58d rest: Specify rest_dispatch handlers by function, not by string.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-22 10:46:28 -07:00
Anders Kaseorg faf600e9f5 urls: Remove unused URL names and shorten others.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-22 10:46:28 -07:00
Mateusz Mandera f078509105 saml: Extract logic for determining if subdomain is allowed by an attr.
This logic has some trickiness due to the need to handle root subdomain
aliases, so this is worth extracting into a helper for use in other
backends.
2020-09-21 22:58:59 -07:00
Mateusz Mandera a0dea84edb saml: Add defensive code against org_membership attr not being a list. 2020-09-21 22:58:59 -07:00
Alex Vandiver e5f62d083e tornado: Merge the TORNADO_SERVER and TORNADO_PORTS configs.
Having both of these is confusing; TORNADO_SERVER is used only when
there is one TORNADO_PORT.  Its primary use is actually to be _unset_,
and signal that in-process handling is to be done.

Rename to USING_TORNADO, to parallel the existing USING_RABBITMQ, and
switch the places that used it for its contents to using
TORNADO_PORTS.
2020-09-21 15:36:16 -07:00
Alex Vandiver 2a12fedcf1 tornado: Remove explicit tornado_processes setting; compute it.
We can compute the intended number of processes from the sharding
configuration.  In doing so, also validate that all of the ports are
contiguous.

This removes a discrepancy between `scripts/lib/sharding.py` and other
parts of the codebase about if merely having a `[tornado_sharding]`
section is sufficient to enable sharding.  Having behaviour which
changes merely based on if an empty section exists is surprising.

This does require that a (presumably empty) `9800` configuration line
exist, but making that default explicit is useful.

After this commit, configuring sharding can be done by adding to
`zulip.conf`:

```
[tornado_sharding]
9800 =              # default
9801 = other_realm
```

Followed by running `./scripts/refresh-sharding-and-restart`.
2020-09-18 15:13:40 -07:00
Alex Vandiver 1928696829 tests: Remove three references to Casper in the comments. 2020-09-18 15:13:40 -07:00
Alex Vandiver 922940e223 tornado: Be more explicit about tornado configuration in testing.
Passing this through the environment needlessly complicates the logic.
2020-09-18 15:13:40 -07:00
Alex Vandiver f638518722 tornado: Move default production port to 9800.
In development and test, we keep the Tornado port at 9993 and 9983,
respectively; this allows tests to run while a dev instance is
running.

In production, moving to port 9800 consistently removes an odd edge
case, when just one worker is on an entirely different port than if
two workers are used.
2020-09-18 15:13:40 -07:00
Dinesh 1dc44703f1 auth: Remove redundant call to pad_method_dict().
This was called in both if and else with the same argument.
I believe there's no reason for it to exist twice and having
it just once would be a bit cleaner.
2020-09-17 13:41:41 -07:00