Commit Graph

1523 Commits

Author SHA1 Message Date
Tim Abbott 15e0304d95 settings: Compute STATIC_ROOT using absolute paths.
This makes it possible to use absolute paths to commands in a
development environment.
2018-12-06 11:05:40 -08:00
Jack Zhang c26f5c6ee3 Rename `check_compatibility` to `check_global_compatibility`. 2018-12-04 17:56:12 -08:00
Jack Zhang ce95973c22 Expand the comment contextualizing the /compatibility endpoint. 2018-12-04 17:56:12 -08:00
Anders Kaseorg 9ba860b4f2 Avoid double redirects to /login and then to /login/.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-12-04 16:12:58 -08:00
Vishnu Ks 788b98d041 portico: Add page for redirecting to a realm subdomain. 2018-12-04 09:35:35 -08:00
Tim Abbott 74007e7f00 bots: Make REMINDER_BOT still a valid setting.
This makes it possible to still run the deliver_scheduled_messages
queue worker, even though we're not creating reminder-bot by default
in new organizations.
2018-12-03 11:43:27 -08:00
Tim Abbott 18e2118fc8 Revert "sessions: Stop using the cached_db sessions engine."
This reverts commit 2fa77d9d54.

Further investigation has determined that this did not fix the
password-reset problem described in the previous commit message;
meanwhile, it causes other problems.  We still need to track down the
root cause of the original password-reset bug.
2018-11-30 15:54:32 -08:00
Raymond Akornor d00b889402 auth: Add an organization reactivation flow with admin confirmation.
This adds a web flow and management command for reactivating a Zulip
organization, with confirmation from one of the organization
administrators.

Further work is needed to make the emails nicer (ideally, we'd send
one email with all the admins on the `To` line, but the `send_email`
library doesn't support that).

Fixes #10783.

With significant tweaks to the email text by tabbott.
2018-11-27 10:58:34 -08:00
Tim Abbott 7b930124d9 push notifications: Add a logger (default-off in tests).
This should suppress some spammy logging output about push
notifications that we were seeing in a large number of unit tests.
2018-11-27 09:45:45 -08:00
Shubham Padia 79e590f50f auth: Store realm id in return_data of social_associate_user_helper.
Realm object is not json-serializable; store the realm id instead
and retrieve the realm in social_auth_finish using
`Realm.objects.get(id=return_data["realm_id"])`.
2018-11-16 12:16:21 -08:00
Shubham Padia d95364b94f auth: GitHubAuthBackend.get_verified_emails returns user's all emails.
The email_list returned has the primary email as the first element.
Testing: The order of the emails in the test was changed to put a
verified email before the primary one. The tests would fail without
this commit's change after the changes in the order of test emails.
2018-11-16 12:16:21 -08:00
Tim Abbott 2fa77d9d54 sessions: Stop using the cached_db sessions engine.
See https://github.com/zulip/zulip/issues/10856 for details on the bug
here; but basically, users who reset their password were unable to
login until the next time we flushed memcached.  The issue disappeared
after stopping using the Django cached_db session engine, so it's
pretty clear that some sort of bug with that session engine
interacting with our password reset logic is the root cause.

Further debugging is required to understand this fully, but for now,
it seems wise to just disable the backend.

The cost of doing so is a small performance decrease, which is likely
acceptable until we can resolve this (it's certainly a more minor
problem than the "Can't login" bug that disabling this removes).
2018-11-16 12:03:21 -08:00
Tim Abbott 106e5c3ed9 docs: Further document tokenized noreply email addresses.
We should still extend email.html to explain the security issue a bit
more clearly, since the article we link to is super long.
2018-11-08 15:26:21 -08:00
Tim Abbott 86b38b1de7 tornado: Ensure TORNADO_PROCESSES is an integer.
When reading something from get_config, the default return type is a
string, which mean that various comparisons didn't work correctly.
2018-11-06 13:08:28 -08:00
Tim Abbott a5acbd51c3 settings: Add new zulip.conf setting for number of Tornado processes.
This will eventually be used to support Tornado sharding; for now,
it's just used to contain the code intended to support that feature.
2018-11-02 16:47:26 -07:00
Tim Abbott ca8e3a278a settings: Extract get_config function.
This is a close analogue to the existing get_secret.
2018-11-02 16:06:46 -07:00
Tim Abbott cf24a20185 tornado: Refactor logic for persistent queue filenames.
Now, these are computed using a function, which will make it easier to
edit these paths to depend on which Tornado process it is in coming
commits.
2018-11-02 14:19:10 -07:00
Rishi Gupta 33d6baa529 portico: Add security doc to /security. 2018-10-30 19:52:24 -07:00
neiljp (Neil Pilgrim) c3cd3e94c1 mypy: Add Optional & check in zproject/backends.py; remove from mypy.ini. 2018-10-29 12:53:16 -07:00
Aditya Bansal f90f701f03 camo: Change CAMO_URI setting value for test suite.
This is a preparatory commit which will help us with removing camo.
In the upcoming commits we introduce a new endpoint which is based
out on the setting CAMO_URI. Since camo could have been hosted on
a different server as well from the main Zulip server, this change
will help us realise in tests how that scenerio might be dealt with.
2018-10-26 16:51:54 -07:00
Tim Abbott 9b0265452f prod_settings_template: Document email mirror needs MX records.
The docs are still pretty much written for expert sysadmins, but this
should help a lot in avoiding confusing about what step is missing.
2018-10-24 15:02:08 -07:00
Tim Abbott a34b79a3f4 python: Avoid importing the mock module in production.
These lazy imports save a significant amount of time on Zulip's core
import process, because mock imports pbr, which in turn import
pkgresources, which is in turn incredibly slow to import.

Fixes part of #9953.
2018-10-17 15:28:48 -07:00
Tim Abbott ec9f6702d8 manage.py: Sabotage importing pika.adapters.twisted_connection.
This is a performance optimization; see the comment.  This fixes part
of #9953.

Eventually, we should do the same thing for importing Tornado as well,
but it's less important because Tornado is a much smaller library.
2018-10-17 11:50:08 -07:00
Kevin Lin b2c29274f3 presence: Allow bots to fetch realm presence data.
Before, presence information for an entire realm could only be queried via
the `POST /api/v1/users/me/presence` endpoint. However, this endpoint also
updates the presence information for the user making the request. Therefore,
bot users are not allowed to access this endpoint because they don't have
any presence data.

This commit adds a new endpoint `GET /api/v1/realm/presence` that just
returns the presence information for the realm of the caller.

Fixes #10651.
2018-10-15 12:51:28 -07:00
Tim Abbott cc066032f6 auth: Add invalid default value for auth_backend_name.
This makes mypy happy with the new logic we just added.
2018-10-11 17:35:07 -07:00
Tim Abbott 20a13b42e6 auth: Automatically maintain list of OAuth backends.
Here, we take advantage of the useful issubclass function to determine
which of our social backends are based on OAuth2.
2018-10-11 17:11:04 -07:00
Tim Abbott 1244a5077e auth: Automate social auth backends in AUTH_BACKEND_NAME_MAP.
This saves a line of manual code every time we add a new social auth
backend.
2018-10-11 17:11:03 -07:00
Tim Abbott b51633efb7 auth: Allow '-' characters in social auth login urls.
This is important for backends like azuread-oauth2.
2018-10-11 16:44:33 -07:00
Tim Abbott 8cf104b643 avatar: Allow API authentication for /avatar/ routes.
This makes it feasibly for the mobile apps to correctly render user
avatars generated by the `!avatar()` syntax.
2018-10-11 15:52:29 -07:00
Steve Howell f1e0152d71 minor: Fix indentation from recent ldap commit.
The linter was complaining.
2018-10-11 15:28:58 -07:00
Vishnu Ks 646ba439f7 settings: Remove unused EMAIL_LOG_DIR setting from dev_settings.
This settings is not used anywhere.
2018-10-11 15:12:08 -07:00
Vishnu Ks 6aa4b64dc0 emails: Don't log emails while running test suite.
Modified the tests to ensure 100% coverage.
2018-10-11 15:12:08 -07:00
Vishnu Ks d8c19cb003 models: Move billing models from zilencer to corporate. 2018-10-11 14:54:29 -07:00
Greg Price d77a4c776d ldap: Skip following dubious referrals.
Some admins setting up Zulip's LDAP auth against Active Directory see
a rather baffling error message: "In order to perform this operation a
successful bind must be completed on the connection".  This happens
despite AUTH_LDAP_BIND_DN and auth_ldap_bind_password being set
perfectly correctly, and on a query that the `ldapsearch` command-line
tool performs quite happily.

Empirically, adding a setting like this to /etc/zulip/settings.py
resolves the issue:
  AUTH_LDAP_CONNECTION_OPTIONS = {
      ldap.OPT_REFERRALS: 0
  }

Some useful, concise background on the LDAP "referral" concept is here:
  https://docs.oracle.com/javase/jndi/tutorial/ldap/referral/overview.html
and a pertinent bit of docs for the underlying Python `ldap` client:
  https://www.python-ldap.org/en/latest/faq.html
and some very helpful documentation for Active Directory:
  https://docs.microsoft.com/en-us/windows/desktop/ad/referrals

Based on the docs above, the story appears to be something like this:

 * This server has the information for part of the scope of our query
   -- in particular it happens to have the information we actually want.

 * But there are other areas ("subordinate domains") that our query is
   in principle asking about, and this server doesn't know if there are
   matches there, so it gives us a referral.

 * And by default, python-ldap lets `libldap` run ahead and attempt to
   bind to those referrals and do those queries too -- which raises an
   error because, unlike Microsoft's "LDAP API", it doesn't reuse the
   credentials.

So if we simply skip trying to follow the referrals, there's no
error... and we already have, from the original response, the answer
we actually need.  That's what the `ldap.OPT_REFERRALS` option does.

There may be more complex situations where the referral really is
relevant, because the desired user info is split across servers.  Even
then, unless an anonymous query will be acceptable, there's no point
in letting `libldap` follow the referral and setting this option is
still the right thing.  When someone eventually comes to this bridge,
some code will be required to cross it, by following the referrals.
That code might look a bit like this (unfinished) example:
  https://bugs.launchpad.net/nav/+bug/1209178

Manually tested by tabbott.

Fixes #343, which was effectively a report of the need for this
OPT_REFERRALS setting.

Fixes #349, since with this change, we no longer require tricky manual
configuration to get Active Directory up and running.
2018-10-11 14:52:13 -07:00
Vishnu Ks d8ddb7f69b settings: Remove reminder-bot from REALM_INTERNAL_BOTS. 2018-10-02 11:12:07 -07:00
Vishnu Ks 6914ee126c billing: Move billing-related views and urls to corporate. 2018-10-01 15:43:21 -07:00
Tim Abbott 923ecb1851 docs: Clean up documentation for fakeldap testing in development. 2018-09-27 13:38:14 -07:00
Greg Price a612049ac4 docs: Expand on LDAP config; try to clarify "username" concept.
The term "username" confusingly refers both to the Django concept of
"username" (meaning "the name the user types into the login form") and
a concept the admin presumably already has in their existing
environment; which may or may not be the same thing, and in fact this
is where we document the admin's choice of whether and how they should
correspond.  The Django concept in particular isn't obvious, and is
counterintuitive when it means something like an email address.

Explicitly explain the Django "username" concept, under the name of
"Zulip username" to take responsibility for our choice of how it's
exposed in the settings interface.  Then use an explicit qualifier,
like "LDAP username", whenever referring to some other notion of
username.  And make a pass over this whole side of the instructions,
in particular for consistent handling of these concepts.
2018-09-27 13:20:10 -07:00
Greg Price 6f23d2f691 docs: Make a pass over LDAP config/setup docs.
Expand on a few things that tend to confuse people (especially the
`%(user)s` thing); move the `LDAPSearchUnion` example out to docs;
adjust the instructions to fit a bit better in their new docs/ home.
2018-09-27 13:20:10 -07:00
Greg Price e7737c6342 docs: Move prose LDAP instructions out of settings template.
This makes it easier to iterate on these, and to expand supplemental
information (like troubleshooting, or unusual configurations) without
further straining the already-dauntingly-long settings.py.

It also makes it easier to consult the instructions while editing the
secrets file, or testing things, etc. -- most admins will find it more
natural to keep a browser open somewhere than a second terminal.
2018-09-27 13:20:10 -07:00
Luis Silva 5cc0fb5c40 auth: Add example using django LDAPSearchUnion function.
This makes it clear how to do a union search, without having to look
anything up.
2018-09-21 11:04:40 -07:00
Vishnu Ks 9489ce0efc plans: Redirect to login if /plans is accessed from realm subdomain. 2018-09-05 13:44:35 -07:00
Shubham Padia 69bfa8c432 auth: Use different defaults for name and email for fakeldap.
Fixes part of #10297.
Use FAKE_LDAP_NUM_USERS which specifies the number of LDAP users
instead of FAKE_LDAP_EXTRA_USERS which specified the number of
extra users.
2018-08-31 17:07:02 +05:30
Roman Godov a2f407c11b api: Add an endpoint for listing the user groups in realm.
Adds endpoint for listing the user groups in realm.
Provides "description", "members", "name" and "id" for each user group.

Fixes #10298
2018-08-27 16:56:23 -07:00
Lyla Fischer 1efcdfdb5c templates: Merge Help and API doc main into documentation_main.
With minor fixes by eeshangarg.
2018-08-27 20:19:24 -02:30
Vishnu Ks 62b93d551a signup: Fix broken resend email link in realm creation flow.
Also use name for selecting form in casper tests
as form with action=new is present in both /new
and /accounts/new/send_confirm/ which breaks
test in CircleCI as
waitWhileVisible('form[action^="/new/"]) never stops
waiting.
2018-08-26 22:54:05 -07:00
Vishnu Ks d2e4417a72 urls: Separate endpoint for signup and new realm email confirm.
This is preparation for the next commit.
2018-08-26 22:53:57 -07:00
Harshit Bansal 96f00d1dfa settings: Remove unused emoji sprite CSS entries in static assets pipeline. 2018-08-26 22:16:23 -07:00
Vishnu Ks 9bb338be11 models: Add plan_type to Realm. 2018-08-21 12:39:06 -07:00
Tim Abbott 3cfb2000cc ldap: Improve error message for username/LDAP domain mismatches. 2018-08-20 10:39:34 -07:00