Commit Graph

9895 Commits

Author SHA1 Message Date
Shubham Padia 80a3651cf3 auth: Let user choose emails in GitHub auth.
Previously, our Github authentication backend just used the user's
primary email address associated with GitHub, which was a reasonable
default, but quite annoying for users who have several email addresses
associated with their GitHub account.

We fix this, by adding a new screen where users can select which of
their (verified) GitHub email addresses to use for authentication.

This is implemented using the "partial" feature of the
python-social-auth pipeline system.

Each email is displayed as a button. Clicking on that button chooses
the email. The email value is stored in a hidden input above the
button. The `primary_email` is displayed on top followed by
`verified_non_primary_emails`. Backend name is also passed as
`backend` to the template, which in our case is GitHub.

Fixes #9876.
2019-06-23 21:27:04 -07:00
Hemanth V. Alluri 720c866e06 devtools: Tweak the integrations tool to use http headers fixtures.
Now that we store HTTP headers in a way that is easy to retreive
by specifying the integration name and fixture name, we should
use it to pre-load the "Custom HTTP Headers" field in the
integrations dev panel.
2019-06-20 16:40:42 -07:00
Hemanth V. Alluri 37d90d9add webhooks: Migrate the GitHub integration to new http headers system.
Idea courtesy of tabbott. See PR #12515 for more details.
2019-06-20 16:37:42 -07:00
Hemanth V. Alluri 7c145c3f68 webhooks: Add a system for storing http headers for integrations.
Using this system, we can now associate any fixture of any integration
with a particular set of HTTP headers. A helper method called
determine_http_headers was introduced, and the test suite was upgraded
to use determine_http_headers.

Comments and documentation significantly edited by tabbott.
2019-06-20 16:36:38 -07:00
Tim Abbott 1817f657ee models: Rename get_admin_users to get_admin_users_and_bots.
This provides much more clarity that administrative bot users are
included in this query as well.
2019-06-20 14:36:15 -07:00
Tim Abbott 4eb9d67b70 models: Extract get_human_admin_users function.
This function is an alternative to get_admin_users that we use in all
places where we explicitly want only human administrative users (not
administrative bots).  The following commits will rename
get_admin_users for better clarity.
2019-06-20 14:32:30 -07:00
Puneeth Chaganti b530c87d26 create_user: Try to read password file only when param specified.
The argument parser has default empty values set for the options
`--password` and `--password-file`, and this causes the script to try and
read a password file even when the argument was not provided.
2019-06-20 11:04:17 -07:00
Puneeth Chaganti 9576ce589d check_user: Get rid of incorrect encode call for initial password. 2019-06-20 11:04:17 -07:00
Mohit Gupta db3d81613b decorator: Refactor @require_non_guest_human_user decorator.
Rename @require_non_guest_human_user to @require_member_or_admin.
This is a refactor commit prior to introduction of Administrator
Bots.
2019-06-18 17:11:58 -07:00
Tim Abbott fd25ced43c import: Fix check for whether data-user-id is present.
Apparently, the `in` keyword in Beautiful Soup does something different.
2019-06-18 11:13:32 -07:00
Tim Abbott 649e363ee3 import: Fix handling of legacy and wildcard mentions.
Our recently-added code for rewriting user IDs on data import didn't
correctly handle wildcard mentions and mentions generated by very old
versions of Zulip (pre data-user-id).
2019-06-18 10:35:01 -07:00
Tim Abbott 2538f84447 import: Fix bad database query for first_message_id.
The previous query ended up doing an awkward join that did not
guarantee use of the Recipient index on zerver_message, turning a very
fast query into something that could take much longer for a single
stream than the rest of the import combined.
2019-06-18 10:25:00 -07:00
Priyank Patel d7b4de2348 messages: Add support for passing user IDs for pm-with clause.
We also document support for user IDs in the pm-with narrow operator.

Edited by tabbott to document on /api rather than in the /help page.

Fixes part of #9474.
2019-06-17 17:38:51 -07:00
Priyank Patel e3431976ce validator: Add validator for checking string or integer list.
This validator will be used to check if string or user IDs list is
passed in.
2019-06-17 17:33:58 -07:00
Hemanth V. Alluri b8268c66d5 webhooks/bitbucket3: Add the `diagnostics:ping` event.
Note: Unlike with the other events supported by this webhook so far,
to determine this event's event type, we need to use the X-Event-Key
HTTP Header.
2019-06-17 17:33:32 -07:00
Tim Abbott 544f9c74ce export: Use outbox emoji for managing who is exported.
This is a little more unambiguous.
2019-06-17 16:10:28 -07:00
Yashashvi Dave 0b4384f4cd custom profile fields: Extract function `validate_custom_field_data`. 2019-06-17 14:56:02 -07:00
Wyatt Hoodes e7b9b93f84 test-backend: Move `bot_avatar` to `var/<uuid>/test-backend`. 2019-06-17 13:56:34 -07:00
Wyatt Hoodes fbcb05dae6 test-backend: Move `test-log-dir` to `var/<uuid>/test-backend`. 2019-06-17 13:56:34 -07:00
Wyatt Hoodes dd339a57af test-backend: Move import/export ouput into `var/<uuid>/test-backend`. 2019-06-17 13:51:40 -07:00
Eeshan Garg f645de255c webhooks/jira: Avoid UnboundLocalErrors for event handler functions.
If the event key is None, the handler content_func never gets
defined, which leads to an UnboundLocalError. This can be easily
avoided by having a dedicated function that handles the case for
when the event key is None.
2019-06-16 15:19:53 -07:00
Hemanth V. Alluri 5a1043237c realm_plan_type: Add wide organization logo related fields to state.
Namely, here we add the "plan_includes_wide_organization_logo" and
"upgrade_text_for_wide_organization_logo" to the page_params (which
is set in zerver/lib/events.py).

"plan_includes_wide_organization_logo" is True if the plan is not of
the Realm.LIMITED type. We need to add this extra boolean parameter
instead of just using "realm_plan_type" to make things a lot easier
to work with on the frontend side, especially considering that
handlebars won't allow checking for equality in its {{#if}} blocks.
2019-06-14 15:59:28 -07:00
Hemanth V. Alluri 5d8050f822 realm_plan_type: Send an event when the realm's plan type is updated.
When a realm's plan type is updated using "do_change_plan_type" we
notify active users of the realm. This way certain plan features
could be enabled instantaneously for active users.
2019-06-14 15:59:28 -07:00
Hemanth V. Alluri 611808381f webhooks/stripe: Blacklist invoice changes under invoiceitem category. 2019-06-14 15:55:07 -07:00
Hemanth V. Alluri 325b1fd6c4 webhooks/stripe: Modify the invoice.updated event handling logic.
If the invoice was paid then the message should simply be
"Invoice is now paid." with a link to the invoice.

Also, suppress the "status_transitions" and "payment_intent"events.
2019-06-14 15:55:07 -07:00
Hemanth V. Alluri eaa89d47fa webhooks/stripe: Add fixtures for invoice and invoiceitem updates. 2019-06-14 15:55:07 -07:00
Hemanth V. Alluri 1a33d73554 management: Don't prepend "HTTP_" to non-headers in parse_headers.
Django does not prepend "HTTP_" to the "Content-Type" or
"Content-Length" headers. So parse_headers should not do that either.
2019-06-14 15:53:59 -07:00
Wyatt Hoodes 6b033c7909 test-backend: Add steps to deal with potential database leaks.
A function was written in `test_fixtures.py` to drop a test database
template if the corresponding database id doesn't belong to a file.
Alongside this fact, every file that is written is removed after 60
minutes.  Meaning any potential database template can never exist
longer than one hour.

This follow-up work was added to deal with the potential race
conditions when running `test-backend`.  Ensuring that all templates
are properly dealt with.

Essentially rewritten by tabbott for cleanliness.

Fixes the remainder of #12426.
2019-06-14 15:23:25 -07:00
Wyatt Hoodes 0b05d91e62 test_runner: Write database ids to file for reference in clean up.
The ids that will be used for each particular run of the test suite are
written to a unique file. Each file will then be used as a time
reference of when the suite was ran.

This change sets up the ability for a complete clean up of potentially
leaked database templates.

Tweaked by tabbott to remove these files after successful database
cleanup.
2019-06-14 15:23:20 -07:00
Wyatt Hoodes 900d9d3e49 test_runner: Add comment explaining abnormal database destruction.
When running the test-backend suite in serial mode, `destroy_test_db`
double appends the database id number to the template if passed an
argument for `number`.  The comment here explains this behavior.
2019-06-14 14:35:39 -07:00
Vishnu Ks cc91e6cb94 auth: Allow setting GOOGLE_OAUTH2_CLIENT_ID from dev-secrets.
This makes it much more convenient to use Google/GitHub authentication
in a Zulip development environment for testing; one only has to set it
up once.
2019-06-14 10:16:26 -07:00
Vishnu Ks 16a345a766 auth: Document setting social_auth_github_key in dev-secrets.
This is a simpler configuration method for development, because
dev_settings.py isn't something the developer can edit.
2019-06-14 10:16:26 -07:00
vinitS101 a6eda858d0 ldap: Fix avatar sync not working with the S3 backend.
This fixes an issue that caused LDAP synchronization to fail for
avatars.  The problem occurred due to the lack of a 'name' attribute
on the BytesIO object that we pass to the upload backend (which is
only used in the S3 backend for computing Content-Type).

Fixes #12411.
2019-06-13 15:12:13 -07:00
Yago González 57c80669a0 api docs: Remove migrated endpoints from deprecated specs. 2019-06-13 14:31:55 -07:00
Yago González c9920ab902 api docs: Add explanatory note to zulip-2.0.yaml. 2019-06-13 14:31:55 -07:00
Mateusz Mandera 8f15884c7d retention: Delete objects tied to a Message in one query with archiving.
Rather than relying on the CASCADING property of the ForeignKey to the
Message table to clean up these objects, we delete them in the same
query as we archive them - since it's guaranteed that any of these
objects that we archive will be deleted due to their Message being
deleted later.
We don't have this guarantee for Attachment objects, which is why we
can't apply this scheme to them.
2019-06-13 11:18:11 -07:00
Mateusz Mandera 25810752fe retention: Fully process each Message chunk in a transaction.
To ensure the database retains a consistent state if archiving gets
interrupted, we process each Messages chunk together with related
objects in a single atomic transaction.
2019-06-13 11:17:54 -07:00
Mateusz Mandera 55eb46433b retention: Use yield when batching instead of returning a list of lists.
This generator architecture will be cleaner for supporting the
transactionality model we want.
2019-06-13 11:11:34 -07:00
Mateusz Mandera 37a22844b9 retention: Clean up code of move_messages_to_archive(). 2019-06-13 11:02:11 -07:00
Mateusz Mandera a68c460a14 retention: Clean up code for archiving attachment_messages.
We had two duplicate functions for archiving zerver_attachment_messages
rows, doing the same thing - archiving by message_id. One of them had a
redundant INNER JOIN, so we get rid of that too.
2019-06-13 11:02:11 -07:00
Mateusz Mandera cbee5beeac retention: Log progress through the archiving process. 2019-06-13 11:02:11 -07:00
Mateusz Mandera e3c7a5d896 retention: Loop over realms in archive_messages.
Since we loop over realms in the functions for archiving stream messages
and then personal+huddle messages, and also want to split cleaning up
attachments by realm - it makes sense to do it all in one single loop.
2019-06-13 11:02:11 -07:00
Yashashvi Dave 8e269b4651 models: Rename notification to `enable_stream_audible_notifications`.
Rename notification property `enable_stream_sounds` to
`enable_stream_audible_notifications` to match with other
notification property patterns.

Fixes part of #12304
2019-06-12 16:24:51 -07:00
Mateusz Mandera 5b8140cf75 retention: Group stream message archiving by realm.
We group the process of archiving stream message by realm, to allow
logging and keeping track of time taken per realm.
2019-06-11 09:25:25 -07:00
Mateusz Mandera f06a4b4eab retention: Batch Message archiving queries.
We batch queries that archive Messages, to limit the maximum amount of
Message objects archived in a single query. This leads to the archiving
of other related objects being batched as well, because we loop over
chunks of archived messages and archive their related objects per-chunk.
2019-06-11 09:25:25 -07:00
Yashashvi Dave f25e00bcaa zever/lib/validator: Rename function to `validate_choice_field_data`. 2019-06-06 22:36:16 -07:00
Wyatt Hoodes ccfcd6b4d8 test_runner: Fix accumulation of uniquely named test templates.
N = self.parallel templates are created, and these templates were
previously named 'zulip_test_template_<1, N>'.  However, to support
running multiple instances of `test-backend`, a unique
`random_id_range_start` was created for each template database.

There was no problem prior because the templates would simply be
used again and thus did not require any clean up. Now that there are
unique database names being created, every time `test-backend` is run
these templates can accumulate on disk.  Instead, we clean up our
templates at the end of every complete run of the test suite, or upon a
SIGINT.

Fixes: #12426
2019-06-06 14:45:32 -07:00
Wyatt Hoodes d3500867d9 test_runner: Fix small typo in comment. 2019-06-06 14:29:57 -07:00
Tim Abbott 1de4b94fbe openapi: Add validation of parameter lists against actual code.
This validation is incomplete, in large part because of the long list
of TODOs in this code.  But this test should provide a ton of support
for us in avoiding regressions as we work towards having complete API
documentation.

See https://github.com/zulip/zulip/issues/12521 for a bunch of
follow-up improvements.
2019-06-06 13:41:07 -07:00
Tim Abbott 065575debf retention: Add a quick comment explaining how deletion works. 2019-06-06 11:41:07 -07:00