Commit Graph

2767 Commits

Author SHA1 Message Date
derAnfaenger c8a5ae753c embedded bots: Consistently use 'storage' instead of 'state.' 2017-11-27 21:05:34 -08:00
Tim Abbott 9645c8b31f integrations: Remove disabled googlesearch integration.
This was moved to another part of the python-zulip-api codebase
because it was unfinished.
2017-11-27 20:50:37 -08:00
Vishnu Ks 985768b2fd registration: Check realm against PreregistrationUser realm.
We would allow a user with a valid invitation for one realm to use it
on a different realm instead.  On a server with multiple realms, an
authorized user of one realm could use this (by sending invites to
other email addresses they control) to create accounts on other
realms. (CVE-2017-0910)

With this commit, when sending an invitation, we record the inviting
user's realm on the PreregistrationUser row; and when registering a
user, we check that the PregistrationUser realm matches the realm the
user is trying to register on.  This resolves CVE-2017-0910 for
newly-sent invitations; the next commit completes the fix.

[greg: rewrote commit message]
2017-11-27 14:58:26 -08:00
Tim Abbott 8b935f4e99 settings: Add setting for SYSTEM_BOT_REALM.
This fixes some subtle JavaScript exceptions we've been getting in
zulipchat.com, caused by the system bot realm there not being "zulip"
interacting with get_cross_realm_users.
2017-11-27 14:46:07 -08:00
Tim Abbott 25fd4c5508 bulk_get_users: Edit the cache keys to make them more unique.
While at this point I was to rewrite this function, this at least
plugs the issues for now.
2017-11-27 14:41:31 -08:00
Tim Abbott 339b67f7af get_cross_realm_dicts: Harden against caching bug in last commit.
This should help protect us from future issues with the way that
`bulk_get_users` does caching.

It's likely that we'll want to further restructure `bulk_get_users` to
not have this base_query code path altogether (since it's kinda
buggy), but I'm going to defer that for a time when we have another
user.
2017-11-27 14:35:50 -08:00
Tim Abbott 646ba5b9e5 bulk_get_users: Fix issues with users in multiple realms.
The previous implementation had a subtle caching bug: because it was
sharing its cache with the `get_user_profile_by_email` cache, if a
user happened to have an email in that cache, we'd return it, even
though that user didn't match `base_query`.

This causes `get_cross_realm_users` to no longer have a problematic
caching bug.
2017-11-27 14:34:45 -08:00
Rohitt Vashishtha 6734b00c0b markdown: Hide URL if message is only an image.
Hides URL if the message content == image url so that sending gifs or
images feels less cluttered. Uses the url_to_a() function to generate
the expected url string for matching.

Fixes #7324.
2017-11-27 13:30:18 -08:00
Tim Abbott 649b7b8b9f internal_send_message: Enforce being passed cross-realm bots.
We include ERROR_BOT in this set, even though it's not technically
cross-realm (it just lives in the admin realm).

This code path does not correctly handle emails that correspond to
multiple accounts (because `get_system_bot` does not).  Since it's
intended to only be used by system bots, we add an appropriate
assertion to ensure it is only used for system bots.
2017-11-26 17:14:23 -08:00
Tim Abbott ae8e203717 actions: Merge internal_prep_message into internal_send_message.
We also document the fact that only system bots are supported here.
2017-11-26 17:14:23 -08:00
Tim Abbott 6952dcbdac embedded bots: Stop using internal_send_message for non-system-bots.
This was causing problems, because internal_send_message assumes that
there is a unique user (across all realms) with the given email
address (which is sorta required to support cross-realm bot messages
the way it does).

With this change, it now, in practice, only sends cross-realm bot
messages.
2017-11-26 17:14:23 -08:00
Tim Abbott 27582782fa embedded bots: Flip around ordering of rate-limiting check. 2017-11-26 17:14:23 -08:00
Tim Abbott 5306a9634d email_mirror: Rewrite to not use internal_send_message.
This was causing problems with the fact that `get_system_bot` now only
works for actual system bot users.
2017-11-26 17:14:23 -08:00
Tim Abbott 2210f627a5 signup: Switch active mirror-dummy users to an AssertionError.
Previously, this was a ValidationError, but that doesn't really make
sense, since this condition reflects an actual bug in the code.

Because this happened to be our only test coverage the ValidationError
catch on line 84 of registration.py, we add nocoverage there for now.
2017-11-26 16:13:45 -08:00
Vishnu Ks b9bc1c2b33 Eliminate get_user_profile_by_email from test_classes. 2017-11-26 15:47:56 -08:00
Tim Abbott 9fcdb6c83a notifications: Use call_consume_in_tests for email sending.
This adds a bit of test coverage with really no downside.
2017-11-26 11:46:07 -08:00
Robert Hönig 0e0a8a2b14 queue processor tests: Call consume by default.
This significantly improves the API for queue_json_publish to not be
overly focused on what the behavior of this function should be in our
unit tests.
2017-11-26 11:45:34 -08:00
Tim Abbott 6968b540c8 mypy: Remove type: ignores that are unnecessary with new mypy.
Mostly these can be removed because the broken LXML stubs were removed
from typeshed in https://github.com/python/typeshed/issues/525.
2017-11-25 10:06:28 -08:00
Vaibhav Sagar 711f7bca3b upload: Fix typo in animated GIF error message.
s/must be have/must have the/.
2017-11-24 09:08:38 -08:00
Andy Perez 0449dc8b8a bugdown: Fix handling of nested fenced math blocks.
Change any fence delimiters (backticks and tildes) to be
interpreted literally inside of math fenced_code blocks.

Fixes #5190.
2017-11-22 12:19:43 -08:00
Harshit Bansal 2e7d0cb4cb digests: Fix new stream links.
Fixes: #7479.
2017-11-22 10:07:36 -08:00
Tim Abbott 8f6b39a1c0 mypy: Fix annotations for json_encoder_for_html. 2017-11-22 01:32:46 -08:00
rht e169bb0954 zerver: Remove simplejson dependency.
Modified by tabbott to put the third-party code in a new file.

Fixes #6970.
2017-11-21 21:51:48 -08:00
Tim Abbott b4e67fac36 bot_lib: Fix storage bug introduced by rebase error. 2017-11-21 21:37:54 -08:00
derAnfaenger e526d0c144 embedded bots: Add views to access state. 2017-11-21 21:10:39 -08:00
derAnfaenger d2af8d4cbd embedded bots: Support multi-entry state modification. 2017-11-21 21:10:39 -08:00
derAnfaenger cbadd3d430 embedded bots: Simplify StateError exception messages. 2017-11-21 21:10:12 -08:00
Tim Abbott 141cf21b86 bulk_create: Fix buggy logic for generating recipients_by_email.
This buggy logic from e1686f427c had
broken do-destroy-rebuild-test-database.

Now that we're not just trying to add the Recipient objects for every
user on the system here to profiles_by_id, we also shouldn't be
processing every Recipeint object on the system.  The fix is simple:
because of the patch we got merged into Django upstream,
recipients_to_create actually has the object IDs added to the
Recipient objects passed into Recipient.objects.bulk_create.

This was missed in manual testing, since it only broke `populate_db
--test-suite`.
2017-11-21 21:07:32 -08:00
rht 3f4bf2d22f zerver/lib: Use python 3 syntax for typing.
Extracted from a larger commit by tabbott because these changes will
not create significant merge conflicts.
2017-11-21 20:56:40 -08:00
rht 561ba33f69 zerver/lib: Use python 3 syntax for typing.
Split by tabbott from a larger commit; this covers a batch of files
with no open PRs touching them.
2017-11-21 20:45:52 -08:00
Eeshan Garg 73d0f1db81 lib/integrations: Remove Integration.doc_context.
An Integration object doesn't need access to the context dict used
to render its doc.md, since the context dict is just passed directly to
render_markdown_path.
2017-11-21 20:39:30 -08:00
Tim Abbott e1686f427c bulk_create: Remove assumption that UserProfiles are globally unique.
This isn't used in production, but that could change in the future,
and the fix to make this limited to a single realm is pretty simple.
2017-11-21 20:23:06 -08:00
Vishnu Ks 766511e519 actions: Mark all messages as read when user unsubscribes from stream.
This fixes a bug where, when a user is unsubscribed from a stream,
they might have unread messages on that stream leak.  While it might
seem to be a minor problem, it can cause significant problems for
computing the `unread_msgs` data structures, since it means we need to
add an extra filter for whether the user is still subscribed, either
in the backend or in the UI.

Fixes #7095.
2017-11-21 20:09:17 -08:00
Tim Abbott 3bfb19b5f3 Convert EmailAuthBackend and LDAPAuthBackend to accept a realm. 2017-11-21 18:23:50 -08:00
Vishnu Ks 34689370cd settings: Create UI and backend for setting signups stream. 2017-11-21 17:39:51 -08:00
Vishnu Ks 3d2c9c6098 models: Replace core team with Realm.INITIAL_PRIVATE_STREAM_NAME. 2017-11-21 17:39:51 -08:00
Vishnu Ks a0275a6257 actions: Send new user messages to signup_notifications_stream. 2017-11-21 17:39:50 -08:00
Rishi Gupta 42652713c8 onboarding: Refactor away setup_initial_private_stream.
It is a confusing name, now that the stream creation part has been taken
away.
2017-11-21 17:39:50 -08:00
Vishnu Ks 3d17897563 onboarding: Move signup notifications stream creation to do_create_realm. 2017-11-21 17:39:50 -08:00
Tim Abbott 29408d24ff emoji: Rework reactions validation to have a single function.
This feels more readable to me, and also identified a couple cases
where we were missing test coverage.
2017-11-20 15:27:21 -08:00
Harshit Bansal 87b523f3ef emoji.py: Refactor emoji data consistency checking functions.
Inorder to provide more explicit error messages I have merged the
`emoji_code_is_valid()` and `emoji_name_is_valid()` functions into
`check_emoji_code_consistency()` and `check_emoji_name_consistency()`
respectively.
2017-11-20 15:27:21 -08:00
Tim Abbott 25ddba99f6 lint: Ban general use of user_profile.save().
This often can cause minor caching problems.

Obviously, it'd be better if we had access to the AST and thus could
do this rule for UserProfile objects in general.
2017-11-20 10:57:08 -08:00
Eeshan Garg 9138d3b8d7 integrations: Move integration-specific context to integration_doc.
Instead of populating the context dict with integration-specific
information in render_markdown_path, we now do that in
zerver.views.integrations.integration_doc instead.

Fixes #7401.

Tweaked by tabbott to use cast to handle the typing issues here.
2017-11-19 20:48:46 -08:00
rht 33b1a541d7 zerver/lib: Use python 3 syntax for typing.
With tweaks by tabbott to fix line spacing.
2017-11-18 16:09:04 -08:00
Tim Abbott 81adc92db6 actions: Use get_system_bot for fetching WELCOME_BOT. 2017-11-17 17:25:56 -08:00
Tim Abbott 1ed50ee858 tests: Pass a realm option to login for non-zulip realms.
This better matches the model of how having multiple realms should
work: you need to specify which realm you're logging into.
2017-11-17 15:32:42 -08:00
Greg Price bd5905da5d error_notify: Use `git describe --always`.
This way, even if the repo lacks any tags, we still
at least get the commit ID.
2017-11-17 14:19:20 -08:00
Andy Perez 83f563aeff bugdown: Properly parse empty blockquotes.
The character ">" now only starts a blockquote if the resulting
blockquote would be non-empty.  Thus, by itself, ">" is now
interpreted literally by bugdown, fixing #687.  The message
with contents consisting of ">>>" is now parsed as a doubly
(not triply) nested blockquote with contents ">".  Properly
formed blockquotes have identical behavior as before, but now
bugdown can no longer produce empty blockquotes as output.

Fixes #2886, #687.
2017-11-17 11:41:44 -08:00
derAnfaenger 395f1e9270 embedded bots: Add database config storage.
Storage limititations are only set on the value of
a config entry, since this is the only user-accessible
part of the schema. Keys are statically set by each
embedded bot.
2017-11-16 23:06:38 -08:00
Vishnu Ks 068912ca7d tests: Add test for default stream group signup flow. 2017-11-16 21:17:31 -08:00