Commit Graph

1947 Commits

Author SHA1 Message Date
Anders Kaseorg dbdf67301b memcached: Switch from pylibmc to python-binary-memcached.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-06 12:51:14 -07:00
Mateusz Mandera 15752067dc auth: Treat emails case-insensitively in ExternalAuthResult.
Our intent throughout the codebase is to treat email
case-insensitively.
The only codepath affected by this bug is remote_user_sso, as that's the
only one that currently passes potentially both a user_profile and
ExternalAuthDataDict when creating the ExternalAuthResult. That's why we
add a test specifically for that codepath.
2020-08-05 11:40:35 -07:00
sahil839 ead18b70fd dev_settings: Add EMAIL_PORT setting in zproject/dev_settings.py.
This commit adds EMAIL_PORT setting for explicitly specifying the
port of SMTP provider in dev_settings.py.

We also change email_backends.send_email_smtp to pass EMAIL_PORT
along with EMAIL_HOST to smtplib.SMTP.

After this change, we will not need to include the port along with
host in EMAIL_HOST.

Also updated the email.md docs accordingly for this change.
2020-08-04 11:03:40 -07:00
Clara Dantas f9cbefac41 sentry: Add user role in user_info object.
As part of issue #15344, the error report emails add the user role
information. This commit adds the user role information to be used
by sentry as well.
2020-08-03 13:35:51 -07:00
Hemanth V. Alluri 21c9adbb3c drafts: Add an API endpoint for fetching drafts.
This endpoint will allow a user to fetch their drafts.

Signed-off-by: Hemanth V. Alluri <hdrive1999@gmail.com>
2020-07-28 17:18:35 -07:00
Hemanth V. Alluri 630fd10be1 drafts: Add an API endpoint for deleting drafts.
This endpoint will allow a user to delete a single draft.

Signed-off-by: Hemanth V. Alluri <hdrive1999@gmail.com>
2020-07-28 17:18:35 -07:00
Hemanth V. Alluri 2d307c760e drafts: Add an API endpoint for editing drafts.
This endpoint will allow a user to edit a single draft.

Signed-off-by: Hemanth V. Alluri <hdrive1999@gmail.com>
2020-07-28 17:18:35 -07:00
Hemanth V. Alluri a0f71b7458 drafts: Add an API endpoint for creating drafts.
This endpoint will allow a user to create drafts in bulk.

Signed-off-by: Hemanth V. Alluri <hdrive1999@gmail.com>
2020-07-28 17:18:35 -07:00
Dinesh 9583554d44 auth: Add check_config for apple auth.
Apple has some other obligatory settings other than key and secret.
To handle that this commit adds a function check_config() similar
to that of SAML.
2020-07-28 17:12:49 -07:00
Dinesh ea225bb9b8 auth: Restructure code handling no name data sent by auth backend.
Co-authored-by: Mateusz Mandera <mateusz.mandera@zulip.com>
2020-07-28 17:12:49 -07:00
Dinesh 4afce5d94d apple_auth: Change BUNDLE_ID setting to APP_ID everywhere.
The apple developer webapp consistently refers this App ID. So,
this clears any confusion that can occur.

Since python social auth only requires us to include App ID in
_AUDIENCE(a list), we do that in computed settings making it easier for
server admin and we make it much clear by having it set to
APP_ID instead of BUNDLE_ID.
2020-07-28 17:12:49 -07:00
Dinesh 782970d9f9 apple_auth: Change where private key is stored.
Changes to a better name apple-auth-key.p8 and removes the extra
directory apple.
2020-07-28 17:12:49 -07:00
Dinesh c15d7e3202 requirements: Update social-auth-core to latest version.
Uses git release as this version 3.4.0 is not released to pypi.
This is required for removing some overriden functions of
apple auth backend class AppleAuthBackend.

With the update we also make following changes:

* Fix full name being populated as "None None".
c5c74f27dd that's included in update assigns first_name and last_name
to None when no name is provided by apple. Due to this our
code is filling return_data['full_name'] to 'None None'.
This commit fixes it by making first and last name strings empty.

* Remove decode_id_token override.
Python social auth merged the PR we sent including the changes
we made to decode_id_token function. So, now there is no
necessity for the override.

* Add _AUDIENCE setting in computed_settings.py.
`decode_id_token` is dependent on this setting.
2020-07-28 17:12:49 -07:00
Tim Abbott c1d7e42fa3 settings: Set FAKE_EMAIL_DOMAIN for development environment.
This is necessary to use the development environment with an IP
address EXTERNAL_HOST, which is used in mobile development.
2020-07-28 11:41:06 -07:00
Tim Abbott 3d1a1e0d20 test_logging_handlers: Avoid printing to console.
This lets us test the recursion bug behavior of this logging handler
without resulting in `logging.error` output being printed to the
console in the event that the test passes.
2020-07-27 16:33:36 -07:00
Alex Vandiver 1c8f4b936b sentry: Capture realm name in user information. 2020-07-27 11:07:55 -07:00
Alex Vandiver af046df3be sentry: Allow setting DSN via environment. 2020-07-27 11:07:55 -07:00
Alex Vandiver bfa809181a sentry: Allow reporting errors to sentry.io.
Use the default configuration, which catches Error logging and
exceptions.  This is placed in `computed_settings.py` to match the
suggested configuration from Sentry[1], which places it in `settings.py`
to ensure it is consistently loaded early enough.

It is placed behind a check for SENTRY_DSN soas to not incur the
additional overhead of importing the `sentry_sdk` modules if Sentry is
not configured.

[1] https://docs.sentry.io/platforms/python/django/
2020-07-27 11:07:55 -07:00
Mateusz Mandera 9d1f77edc5 saml: Use self.logger in get_issuing_idp.
get_issuing_idp is no longer a class method, so that akward logger
fetching can be skipped and self.logger can be accessed.
2020-07-26 15:48:48 -07:00
Mateusz Mandera 389c190d49 saml: Fix incorrect settings object being passed in get_issuing_idp.
Fixes #15904.

settings is supposed to be a proper OneLogin_Saml2_Settings object,
rather than an empty dictionary. This bug wasn't easy to spot because
the codepath that causes this to demonstrate runs only if the
SAMLResponse contains encrypted assertions.
2020-07-26 15:48:48 -07:00
Tim Abbott 29c66cf7c2 actions: Remove log_event and its legacy settings.
Now that we've finally converted these to use RealmAuditLog, we can
remove this ultra-legacy bit of code.
2020-07-24 12:13:16 -07:00
Mohit Gupta 7bbba74d95 loggers: Set propagate False for zulip.slow_queries logger.
This will prevent it to propagating to root logger and causing flaky
behavior in tests which verify logs by root logger using assertLogs.
2020-07-22 17:12:28 -07:00
Tim Abbott c83df88aeb help: Improve documentation for Gitter import. 2020-07-22 16:08:48 -07:00
Mateusz Mandera 6a50911032 auth: Allow signing in to an existing account with noreply github email.
In particular importing gitter data leads to having accounts with these
noreply github emails. We generally only want users to have emails that
we can actually send messages to, so we'll keep the old behavior of
disallowing sign up with such an email address. However, if an account
of this type already exists, we should allow the user to have access to
it.
2020-07-22 15:50:43 -07:00
Mateusz Mandera 82660de591 ldap: Use zulip.ldap logger in ZulipLDAPUserPopulator. 2020-07-20 11:01:03 -07:00
Tim Abbott 6be3fca037 auth: Remove short_name from LDAP API.
As best I can tell, we fetched this field and then ignored it, so
unlike the last few commits, this is more a code cleanup than a
functional change.
2020-07-17 11:28:08 -07:00
Steve Howell c44500175d database: Remove short_name from UserProfile.
A few major themes here:

    - We remove short_name from UserProfile
      and add the appropriate migration.

    - We remove short_name from various
      cache-related lists of fields.

    - We allow import tools to continue to
      write short_name to their export files,
      and then we simply ignore the field
      at import time.

    - We change functions like do_create_user,
      create_user_profile, etc.

    - We keep short_name in the /json/bots
      API.  (It actually gets turned into
      an email.)

    - We don't modify our LDAP code much
      here.
2020-07-17 11:15:15 -07:00
Anders Kaseorg c2f9db4602 logo: Update Zulip logo.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-16 01:37:08 -07:00
Vishnu KS 788b6a47e8 email_log: Show reply_to address if present. 2020-07-14 00:36:31 -07:00
Rohitt Vashishtha 912e372c4e markdown: Remove !avatar() and !gravatar() syntax.
This particular commit has been a long time coming. For reference,
!avatar(email) was an undocumented syntax that simply rendered an
inline 50px avatar for a user in a message, essentially allowing
you to create a user pill like:

`!avatar(alice@example.com) Alice: hey!`

---

Reimplementation

If we decide to reimplement this or a similar feature in the future,
we could use something like `<avatar:userid>` syntax which is more
in line with creating links in markdown. Even then, it would not be
a good idea to add this instead of supporting inline images directly.

Since any usecases of such a syntax are in automation, we do not need
to make it userfriendly and something like the following is a better
implementation that doesn't need a custom syntax:

`![avatar for Alice](/avatar/1234?s=50) Alice: hey!`

---

History

We initially added this syntax back in 2012 and it was 'deprecated'
from the get go. Here's what the original commit had to say about
the new syntax:

> We'll use this internally for the commit bot.  We might eventually
> disable it for external users.

We eventually did start using this for our github integrations in 2013
but since then, those integrations have been neglected in favor of
our GitHub webhooks which do not use this syntax.

When we copied `!gravatar` to add the `!avatar` syntax, we also noted
that we want to deprecate the `!gravatar` syntax entirely - in 2013!

Since then, we haven't advertised either of these syntaxes anywhere
in our docs, and the only two places where this syntax remains is
our game bots that could easily do without these, and the git commit
integration that we have deprecated anyway.

We do not have any evidence of someone asking about this syntax on
chat.zulip.org when developing an integration and rightfully so- only
the people who work on Zulip (and specifically, markdown) are likely
to stumble upon it and try it out.

This is also the only peice of code due to which we had to look up
emails -> userid mapping in our backend markdown. By removing this,
we entirely remove the backend markdown's dependency on user emails
to render messages.

---

Relevant commits:

- Oct 2012, Initial commit        c31462c278
- Nov 2013, Update commit bot     968c393826
- Nov 2013, Add avatar syntax     761c0a0266
- Sep 2017, Avoid email use       c3032a7fe8
- Apr 2019, Remove from webhook   674fcfcce1
2020-07-07 10:39:44 -07:00
Anders Kaseorg 5693f63bf8 settings: Convert variable type annotations to Python 3.6 style.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-06 18:44:58 -07:00
Anders Kaseorg 9900298315 zthumbor: Remove Python 2 residue.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-06 18:44:58 -07:00
arpit551 01f12b9fc2 audit_log: Log acting_user in user creation and user activation. 2020-07-06 17:32:09 -07:00
arpit551 653928bdfe audit_log: Log acting_user in do_change_avatar_fields. 2020-07-06 17:24:18 -07:00
Mateusz Mandera d51afcf485 emails: Improve handling of timeouts when sending.
We use the EMAIL_TIMEOUT django setting to timeout after 15s of trying
to send an email. This will nicely lead to retries in the email_senders
queue, due to the retry_send_email_failures decorator.

smtlib documentation suggests that socket.timeout can be raised as the
result of timing out, so in attempts I'm getting
smtplib.SMTPServerDisconnected. Either way, seems appropriate to add
socket.timeout to the exception that we catch.
2020-07-03 16:52:50 -07:00
Vishnu KS 4c6350fa4b billing: Add option to request a sponsorship in /upgrade. 2020-07-01 16:45:38 -07:00
Tim Abbott 08cbb718fa prod_settings_template: Fix confusing {random_token} in comment.
This was a typo in the original implementation; it should say {token}.
2020-07-01 12:25:21 -07:00
Anders Kaseorg 229e08c1d9 backends: Fix a type: ignore issue.
I checked that this does not interfere with the MRO of the auth
backends:

In [1]: import zproject.backends; zproject.backends.GitHubAuthBackend.__mro__
Out[1]:
(zproject.backends.GitHubAuthBackend,
 zproject.backends.SocialAuthMixin,
 zproject.backends.ZulipAuthMixin,
 zproject.backends.ExternalAuthMethod,
 abc.ABC,
 social_core.backends.github.GithubOAuth2,
 social_core.backends.oauth.BaseOAuth2,
 social_core.backends.oauth.OAuthAuth,
 social_core.backends.base.BaseAuth,
 object)

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-30 18:58:23 -07:00
Aman Agrawal 5b7917da5f notifications: Remove support for unbatched push removal events.
We remove support for the old clients which required an event for
each message to clear notification.

This is justified since it has been around 1.5 years since we started
supporting the bulk operation (and so essentially nobody is using a
mobile app version so old that it doesn't support the batched
approach) and the unbatched approach has a maintenance and reliability
cost.
2020-06-30 10:12:27 -07:00
Anders Kaseorg 7f46886696 settings: Split hostname from port more carefully.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-29 22:19:47 -07:00
Steve Howell 69be97e365 pointer: Remove pointer from API and page_params.
There is still some miscellaneous cleanup that
has to happen for things like analytics queries
and dead code in node tests, but this should
remove the main use of pointers in the backend.

(We will also still need to drop the DB field.)
2020-06-27 16:44:38 -07:00
Tim April 8e2a79095d mobile: Add support for alternative mobile URI.
Due to authentication restrictions, a deployment may need to direct
traffic for mobile applications to an alternate uri to take advantage
alternate authentication mechansism. By default the standard realm URI
will be usedm but if overridden in the settings file, an alternate uri
can be substituted.
2020-06-26 12:13:26 -07:00
Mateusz Mandera be4aa65827 auth: Fix incorrect encoding of whitespace in scope in Apple backend. 2020-06-25 13:45:42 -07:00
Tim Abbott cb1321d0d2 lint: Harden various checks for URLs.
Because of other validation on these values, I don't believe any of
these does anything different, but these changes improve readability
and likely make GitHub's code scanners happy.
2020-06-25 12:10:45 -07:00
Tim Abbott 3bdf4ef21f Revert "search: Enable pills-based UI for testing."
This reverts commit 90e180f974.

This was meant to stay on the chat.zulip.org branch.
2020-06-24 16:58:58 -07:00
Tim Abbott 90e180f974 search: Enable pills-based UI for testing. 2020-06-24 10:44:37 -07:00
Greg Price 0021dfe8e7 auth settings: Put Apple "Team ID" before the things it namespaces.
I believe the Bundle ID (aka App ID) and Services ID have meaning only
relative to a specific Team ID.  In particular, in some places in the
developer.apple.com UI, they're displayed in a fully-qualified form
like "ABCDE12345.com.example.app", where "com.example.app" is the
App ID or Services ID and ABCDE12345 is the Team ID.
2020-06-23 19:04:26 -07:00
Brainrecursion 30eaed0378 saml: Add option to restrict subdomain access based on SAML attributes.
Adds the ability to set a SAML attribute which contains a
list of subdomains the user is allowed to access. This allows a Zulip
server with multiple organizations to filter using SAML attributes
which organization each user can access.

Cleaned up and adapted by Mateusz Mandera to fit our conventions and
needs more.

Co-authored-by: Mateusz Mandera <mateusz.mandera@zulip.com>
2020-06-23 17:14:31 -07:00
wowol fec77381ee urls: Migrate zproject urls to use modern django patterns.
This migrations use of url() to path() or re_path().  In this commit,
we only migration regular expressions to path where the translation is trivial:
* URLs with no parameters in them
* URLs with only integer parameters in them
* Strings where there regular expression just checked for `/`s
path; strings, which can have variable validation in the URLs that
need by-hand auditing, we leave for future commits that are easier to
review and think about the individual changes.

Modified by tabbott to convert back to `re_path` various URLs with
strings that had been converted to use `path()` with string
validation to simplify review.

Fixes #14770.
2020-06-23 15:23:09 -07:00
wowol a3c475f137 urls: Migrate dev urls to use modern django patterns. 2020-06-23 15:02:44 -07:00