Commit Graph

1546 Commits

Author SHA1 Message Date
Aditya Bansal 079dfadf1a camo: Add endpoint to handle camo requests.
This endpoint serves requests which might originate from an image
preview link which had an http url and the message holding the image
link was rendered before we introduced thumbnailing. In that case
we would have used a camo proxy to proxy http content over https and
avoid mix content warnings.

In near future, we plan to drop use of camo and just rely on thumbor
to serve such images. This endpoint helps maintain backward
compatibility for links which were already rendered.
2019-01-04 10:27:04 -08:00
Aditya Bansal 3ee69f3da9 thumbnails: Add setting THUMBNAIL_IMAGES.
This setting splits away part of responsibility from THUMBOR_URL.
Now on, this setting will be responsible for controlling whether
we thumbnail images or not by asking bugdown to render image links
to hit our /thumbnail endpoint. This is irrespective of what
THUMBOR_URL is set to though ideally THUMBOR_URL should be set
to point to a running thumbor instance.
2019-01-04 10:27:04 -08:00
Steve Howell 423db23c36 status: Add endpoint to toggle "away" status. 2019-01-02 09:16:30 -08:00
Tim Abbott f51ca9f398 backends: Don't try to process userAccountControl unless configured.
This fixes an exception in manage.py sync_ldap_user_data if
userAccountControl is not setup on the system yet.
2018-12-30 11:05:14 -08:00
Tim Abbott 331984c322 ldap: Cast account_control_values to int.
This value will usually apparently come through the LDAP API as a
string, apparently.
2018-12-29 16:35:13 -08:00
Tim Abbott 772026e66b ldap: Don't crash if some users don't have a thumbnailPhoto.
It's normal for an LDAP database to have some users with a
thumbnailPhoto field set and others without one, so we should support
this configuration.
2018-12-29 16:32:16 -08:00
Rishi Gupta 84a31ca800 billing: Remove BillingProcessor.
Leaving the model in place, so that we can do the database migration by
hand.
2018-12-22 13:30:15 -08:00
Tim Abbott 9c3f38a564 docs: Automatically construct OpenAPI metadata for help center.
This is somewhat hacky, in that in order to do what we're doing, we
need to parse the HTML of the rendered page to extract the first
paragraph to include in the open graph description field.  But
BeautifulSoup does a good job of it.

This carries a nontrivial performance penalty for loading these pages,
but overall /help/ is a low-traffic site compared to the main app, so
it doesn't matter much.

(As a sidenote, it wouldn't be a bad idea to cache this stuff).

There's lots of things we can improve in this, largely through editing
the articles, but we can deal with that over time.

Thanks to Rishi for writing all the tests.
2018-12-19 10:18:20 -08:00
Tim Abbott 0510424e1c auth: Remove some now-unused auth_backend_enabled helpers.
These were causing coverage errors, and in any case are now useless.
2018-12-18 16:51:57 -08:00
seresheim 49dbd85a89 auth: Add support for Azure Active Directory authentication.
This takes advantage of all of our work on making the
python-social-auth integration reusable for other authentication
backends.
2018-12-18 16:39:03 -08:00
Joshua Pan ad1df0ebeb settings: Add support for customizing the top-left logo.
This adds a new realm_logo field, which is a horizontal-format logo to
be displayed in the top-left corner of the webapp, and any other
places where we might want a wide-format branding of the organization.

Tweaked significantly by tabbott to rebase, fix styling, etc.

Fixing the styling of this feature's loading indicator caused me to
notice the loading indicator for the realm_icon feature was also ugly,
so I fixed that too.

Fixes #7995.
2018-12-18 12:44:52 -08:00
Tim Abbott 142d9cb63f report: Allow error-reporting views from unauthed users.
This should make it possible for blueslip error reports to be sent on
our logged-out portico pages, which should in turn make it possible to
debug any such issues as they occur.
2018-12-16 15:44:48 -08:00
Rishi Gupta 111eda604b portico: Add /atlassian for Zulip users migrating from HipChat. 2018-12-14 23:42:47 -08:00
Tim Abbott 626e191201 ldap: Add support for automatic user deactivation/reactivation.
As part of this, extend our documentation on synchronizing data from
Active Directory.
2018-12-13 16:24:15 -08:00
Tim Abbott 0a5221a819 ldap: Extract dev_ldap_directory.py.
This gets what is fundamentally unit testing code out of backends.py.
2018-12-13 16:24:15 -08:00
Tim Abbott 5dd646f33f ldap: Add support for syncing avatar images from LDAP.
This should make life a lot more convenient for organizations that use
the LDAP integration and have their avatars in LDAP already.

This hasn't been end-to-end tested against LDAP yet, so there may be
some minor revisions, but fundamentally, it works, has automated
tests, and should be easy to maintain.

Fixes #286.
2018-12-13 13:39:22 -08:00
Tim Abbott ee901ac8b1 django: Remove the Django contrib Sites app to fix 2FA QR codes.
Apparently, Django's get_current_site function (used, e.g., in
django-two-factor to look up the domain to use in QR codes) first
tries to use the Sites framework, and if unavailable, does the right
thing (namely, using request.get_host()).

We don't use the Sites framework for anything in Zulip, so the correct
fix is to just remove it.

Fixes #11014.
2018-12-12 15:01:55 -08:00
Tim Abbott 8a11c94a2d fakeldap: Add thumbnailPhoto/jpegPhoto for testing avatar syncing. 2018-12-12 11:23:33 -08:00
Tim Abbott b5e65a2ea0 fakeldap: Move fakeldap configuration into ZulipLDAPAuthBackendBase.
This allows us to use this for testing the ZulipLDAPUserPopulator code
as well.
2018-12-12 11:07:05 -08:00
Tim Abbott 33fb750275 fakeldap: Invert order of loop/conditionals.
This is about to save us some work when we add image attributes.
2018-12-12 10:46:54 -08:00
Roman Godov 9c8eeaed85 digest_email: Add endpoint for rendering digest to the web.
Adds "/digest/" endpoint for rendering content of digest email
to the web.

Fixes #9974
2018-12-11 13:38:30 -08:00
Jack Zhang be9b6a6dee compatibility: Add a compatibility check to api_get_server_settings.
This should make it convenient for the mobile app to present errors of
the form "Your Zulip app is not new enough for this Zulip server".
2018-12-10 15:06:09 -08:00
Tim Abbott e603237010 email: Convert accounts code to use delivery_email.
A key part of this is the new helper, get_user_by_delivery_email.  Its
verbose name is important for clarity; it should help avoid blind
copy-pasting of get_user (which we'll also want to rename).
Unfortunately, it requires detailed understanding of the context to
figure out which one to use; each is used in about half of call sites.

Another important note is that this PR doesn't migrate get_user calls
in the tests except where not doing so would cause the tests to fail.
This probably deserves a follow-up refactor to avoid bugs here.
2018-12-06 16:21:38 -08:00
Tim Abbott 15e0304d95 settings: Compute STATIC_ROOT using absolute paths.
This makes it possible to use absolute paths to commands in a
development environment.
2018-12-06 11:05:40 -08:00
Jack Zhang c26f5c6ee3 Rename `check_compatibility` to `check_global_compatibility`. 2018-12-04 17:56:12 -08:00
Jack Zhang ce95973c22 Expand the comment contextualizing the /compatibility endpoint. 2018-12-04 17:56:12 -08:00
Anders Kaseorg 9ba860b4f2 Avoid double redirects to /login and then to /login/.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-12-04 16:12:58 -08:00
Vishnu Ks 788b98d041 portico: Add page for redirecting to a realm subdomain. 2018-12-04 09:35:35 -08:00
Tim Abbott 74007e7f00 bots: Make REMINDER_BOT still a valid setting.
This makes it possible to still run the deliver_scheduled_messages
queue worker, even though we're not creating reminder-bot by default
in new organizations.
2018-12-03 11:43:27 -08:00
Tim Abbott 18e2118fc8 Revert "sessions: Stop using the cached_db sessions engine."
This reverts commit 2fa77d9d54.

Further investigation has determined that this did not fix the
password-reset problem described in the previous commit message;
meanwhile, it causes other problems.  We still need to track down the
root cause of the original password-reset bug.
2018-11-30 15:54:32 -08:00
Raymond Akornor d00b889402 auth: Add an organization reactivation flow with admin confirmation.
This adds a web flow and management command for reactivating a Zulip
organization, with confirmation from one of the organization
administrators.

Further work is needed to make the emails nicer (ideally, we'd send
one email with all the admins on the `To` line, but the `send_email`
library doesn't support that).

Fixes #10783.

With significant tweaks to the email text by tabbott.
2018-11-27 10:58:34 -08:00
Tim Abbott 7b930124d9 push notifications: Add a logger (default-off in tests).
This should suppress some spammy logging output about push
notifications that we were seeing in a large number of unit tests.
2018-11-27 09:45:45 -08:00
Shubham Padia 79e590f50f auth: Store realm id in return_data of social_associate_user_helper.
Realm object is not json-serializable; store the realm id instead
and retrieve the realm in social_auth_finish using
`Realm.objects.get(id=return_data["realm_id"])`.
2018-11-16 12:16:21 -08:00
Shubham Padia d95364b94f auth: GitHubAuthBackend.get_verified_emails returns user's all emails.
The email_list returned has the primary email as the first element.
Testing: The order of the emails in the test was changed to put a
verified email before the primary one. The tests would fail without
this commit's change after the changes in the order of test emails.
2018-11-16 12:16:21 -08:00
Tim Abbott 2fa77d9d54 sessions: Stop using the cached_db sessions engine.
See https://github.com/zulip/zulip/issues/10856 for details on the bug
here; but basically, users who reset their password were unable to
login until the next time we flushed memcached.  The issue disappeared
after stopping using the Django cached_db session engine, so it's
pretty clear that some sort of bug with that session engine
interacting with our password reset logic is the root cause.

Further debugging is required to understand this fully, but for now,
it seems wise to just disable the backend.

The cost of doing so is a small performance decrease, which is likely
acceptable until we can resolve this (it's certainly a more minor
problem than the "Can't login" bug that disabling this removes).
2018-11-16 12:03:21 -08:00
Tim Abbott 106e5c3ed9 docs: Further document tokenized noreply email addresses.
We should still extend email.html to explain the security issue a bit
more clearly, since the article we link to is super long.
2018-11-08 15:26:21 -08:00
Tim Abbott 86b38b1de7 tornado: Ensure TORNADO_PROCESSES is an integer.
When reading something from get_config, the default return type is a
string, which mean that various comparisons didn't work correctly.
2018-11-06 13:08:28 -08:00
Tim Abbott a5acbd51c3 settings: Add new zulip.conf setting for number of Tornado processes.
This will eventually be used to support Tornado sharding; for now,
it's just used to contain the code intended to support that feature.
2018-11-02 16:47:26 -07:00
Tim Abbott ca8e3a278a settings: Extract get_config function.
This is a close analogue to the existing get_secret.
2018-11-02 16:06:46 -07:00
Tim Abbott cf24a20185 tornado: Refactor logic for persistent queue filenames.
Now, these are computed using a function, which will make it easier to
edit these paths to depend on which Tornado process it is in coming
commits.
2018-11-02 14:19:10 -07:00
Rishi Gupta 33d6baa529 portico: Add security doc to /security. 2018-10-30 19:52:24 -07:00
neiljp (Neil Pilgrim) c3cd3e94c1 mypy: Add Optional & check in zproject/backends.py; remove from mypy.ini. 2018-10-29 12:53:16 -07:00
Aditya Bansal f90f701f03 camo: Change CAMO_URI setting value for test suite.
This is a preparatory commit which will help us with removing camo.
In the upcoming commits we introduce a new endpoint which is based
out on the setting CAMO_URI. Since camo could have been hosted on
a different server as well from the main Zulip server, this change
will help us realise in tests how that scenerio might be dealt with.
2018-10-26 16:51:54 -07:00
Tim Abbott 9b0265452f prod_settings_template: Document email mirror needs MX records.
The docs are still pretty much written for expert sysadmins, but this
should help a lot in avoiding confusing about what step is missing.
2018-10-24 15:02:08 -07:00
Tim Abbott a34b79a3f4 python: Avoid importing the mock module in production.
These lazy imports save a significant amount of time on Zulip's core
import process, because mock imports pbr, which in turn import
pkgresources, which is in turn incredibly slow to import.

Fixes part of #9953.
2018-10-17 15:28:48 -07:00
Tim Abbott ec9f6702d8 manage.py: Sabotage importing pika.adapters.twisted_connection.
This is a performance optimization; see the comment.  This fixes part
of #9953.

Eventually, we should do the same thing for importing Tornado as well,
but it's less important because Tornado is a much smaller library.
2018-10-17 11:50:08 -07:00
Kevin Lin b2c29274f3 presence: Allow bots to fetch realm presence data.
Before, presence information for an entire realm could only be queried via
the `POST /api/v1/users/me/presence` endpoint. However, this endpoint also
updates the presence information for the user making the request. Therefore,
bot users are not allowed to access this endpoint because they don't have
any presence data.

This commit adds a new endpoint `GET /api/v1/realm/presence` that just
returns the presence information for the realm of the caller.

Fixes #10651.
2018-10-15 12:51:28 -07:00
Tim Abbott cc066032f6 auth: Add invalid default value for auth_backend_name.
This makes mypy happy with the new logic we just added.
2018-10-11 17:35:07 -07:00
Tim Abbott 20a13b42e6 auth: Automatically maintain list of OAuth backends.
Here, we take advantage of the useful issubclass function to determine
which of our social backends are based on OAuth2.
2018-10-11 17:11:04 -07:00
Tim Abbott 1244a5077e auth: Automate social auth backends in AUTH_BACKEND_NAME_MAP.
This saves a line of manual code every time we add a new social auth
backend.
2018-10-11 17:11:03 -07:00