Commit Graph

6015 Commits

Author SHA1 Message Date
Alena Volkova 0e653d198d test_decorators: Replace the json/tutorial_status endpoint in tests.
This endpoint is about to become an API-style route and have the legacy
decorator removed from its view. The json/fetch_api_key endpoint will be
used in tests instead of it.
2017-10-09 15:13:33 -07:00
Steve Howell 75db4f5187 Avoid O(N) query pitfall for /users endpoint.
We now use a `.values` query to get just the fields we need
in order to fulfill '/json/users' requests.

The main benefit is that we don't do O(N) queries for bot
owners, but we also have less data on UserProfile to process.
2017-10-09 14:08:07 -07:00
Steve Howell f87159ad98 Flush standard output in match_states(). 2017-10-09 12:23:49 -07:00
Eeshan Garg 0ca1224b3e integrations: Render xkcd bot's documentation. 2017-10-09 11:40:44 -07:00
Harshit Bansal 3acaa79336 reactions.py: Don't check for valid emoji name while removing reaction.
On receiving a request for deleting a reaction, just check if such
a reaction exists or not. If it exists then just delete the reaction
otherwise send an error message that such a reaction doesn't exist.
It doesn't make sense to check whether an emoji name is valid or not.
2017-10-09 17:54:37 +00:00
Eeshan Garg 71eee35bce webhooks: Add a Google Code-in integration. 2017-10-09 09:04:39 -07:00
Tim Abbott 9cf26db6db test_management_commands: Fix send_webhook_fixture_message test.
I unfortunately didn't remember this could have a test.
2017-10-08 21:20:35 -07:00
Steve Howell a67b07bfc4 Avoid shadowing stream variables.
This helps with mypy typings in an upcoming commit.
2017-10-08 20:18:34 -07:00
Steve Howell 2e43562832 Extract Stream.num_subscribers_for_stream_id.
This allows us to count subscribers without a Stream object.
2017-10-08 20:18:34 -07:00
Steve Howell 12e65eb21c Use stream ids in various tests.
This commit prepares us to introduce a StreamLite class. For
these tests, we don't care about the actual contents of the
Stream, just the right stream is there.
2017-10-08 20:18:34 -07:00
Steve Howell c1d7fc6e80 Only require stream_id in private_stream_user_ids(). 2017-10-08 20:18:34 -07:00
Steve Howell 7dbea8a2bf Only require stream_id in subscribed_to_stream().
Since subscribed_to_stream is only doing an id lookup
on the Stream model to find out if a user is subscribed to
a stream, there's no reason to require a full Stream object.

It's currently the case that all callers do have full Stream
objects handy to pass in to this function, but it's still a
good practice to have functions only ask for objects that they
need.
2017-10-08 20:18:34 -07:00
Tim Abbott 49274a060c send_webhook_fixture_message: Fix to work with subdomains.
We apparently failued to update this tool when we switched the
development environment to require a subdomain.
2017-10-08 20:14:14 -07:00
Greg Price a116303604 passwords: Express the quality threshold as guesses required.
The original "quality score" was invented purely for populating
our password-strength progress bar, and isn't expressed in terms
that are particularly meaningful.  For configuration and the core
accept/reject logic, it's better to use units that are readily
understood.  Switch to those.

I considered using "bits of entropy", defined loosely as the log
of this number, but both the zxcvbn paper and the linked CACM
article (which I recommend!) are written in terms of the number
of guesses.  And reading (most of) those two papers made me
less happy about referring to "entropy" in our terminology.
I already knew that notion was a little fuzzy if looked at
too closely, and I gained a better appreciation of how it's
contributed to confusion in discussing password policies and
to adoption of perverse policies that favor "Password1!" over
"derived unusual ravioli raft".  So, "guesses" it is.

And although the log is handy for some analysis purposes
(certainly for a graph like those in the zxcvbn paper), it adds
a layer of abstraction, and I think makes it harder to think
clearly about attacks, especially in the online setting.  So
just use the actual number, and if someone wants to set a
gigantic value, they will have the pleasure of seeing just
how many digits are involved.

(Thanks to @YJDave for a prototype that the code changes in this
 commit are based on.)
2017-10-08 15:48:44 -07:00
rht fbd24669a4 Cleanup: Remove shebang and executable flag from these files. 2017-10-08 15:38:15 -07:00
Tim Abbott ec080aed6b mypy: Workaround lxml annotations being busted. 2017-10-08 12:38:20 -07:00
Tim Abbott d215ea1e37 actions: Rename all_subs_by_stream to all_subscribers_by_stream.
The previous name sounded a bit too much like they were subcription
objects.
2017-10-08 12:33:53 -07:00
Steve Howell 3e6bfe1b23 Use user_ids, not emails, for bulk stream operations.
We now return user_ids for subscribers to streams in add-stream
events.  This allows us to eliminate the UserLite class for
both bulk adds and bulk removes.  It also simplifies some JS
code that already wanted to use user_ids, not emails.

Fixes #6898
2017-10-08 12:31:12 -07:00
Harshit Bansal 3c434f0d86 notifications: Switch to use `make_links_absolute()` from lxml library.
Instead of using custom regexes for converting relative URLs to
absolute URLs switch to using `make_links_absolute()` function
from lxml library.
2017-10-08 12:15:30 -07:00
Steve Howell 10a30bece1 Rename presence_idle_userids -> presence_idle_user_ids. 2017-10-07 12:16:45 -07:00
Steve Howell fbaef43ac3 Rename bot_owner_userids -> bot_owner_user_ids. 2017-10-07 12:16:45 -07:00
Greg Price aa4104a5af logging: Add option to show the PID in each log message. 2017-10-06 19:21:40 -07:00
Harshit Bansal 7b88e3eea7 tests: Add a test suite for verifying the mobile push notifs content.
This test suite works by using the expected_output and new text_output
fields in the bugdown test cases to verify that each syntax is
correctly translated by this new function.

Some of these translations, like strikethrough, are kinda poor; but
this framework should make it easy to iterate on the formatting.

Fixes: #6720.
2017-10-06 16:47:27 -07:00
Harshit Bansal 5a6584890d push_notifications: Start using `get_mobile_push_content()` function. 2017-10-06 16:47:25 -07:00
Harshit Bansal 28628eeaeb push_notifications: Add `truncate_content()` function.
This function truncates the textual content at correct length.
(It will be updated later to handle corner cases of unicode
combining characters and tags when we start supporting them.)
2017-10-06 16:44:19 -07:00
Harshit Bansal b5a1aacfb3 push_notifications: Add `get_mobile_push_content()` function.
Given the rendered content of a message, this function strips
all the markup replacing emojis with their corresponding unicode
representation.
2017-10-06 16:44:18 -07:00
Tim Abbott 234f7960a1 tests: Add @slow decorator to various methods needing it.
Also, fix the test_message_editing test having way too many cases.

Fixes #1478.
2017-10-06 15:45:04 -07:00
Tim Abbott 0d7617721c test_messages: Clean up some slow tests.
One isn't actually slow; the other was doing something wasteful.
2017-10-06 15:44:17 -07:00
Tim Abbott d5c2508858 tests: Remove unnecessary @slow decorators.
This is part of addressing #1478.
2017-10-06 15:17:34 -07:00
Steve Howell 9202777d7f tests: Provide more useful output in assert_length(). 2017-10-06 14:30:30 -07:00
Steve Howell d2966ae5d4 minor: Fix missing "not" in comment. 2017-10-06 14:08:41 -07:00
Tim Abbott 73e1c26c15 test_events: Add missing nocoverage tag on debugging code. 2017-10-06 13:59:26 -07:00
Steve Howell ad521a0e07 minor: Add debugging statement to test.
This test is flaky for some unknown reason, so we print
out the loop index in case the test fails.
2017-10-06 13:56:37 -07:00
Steve Howell a99574ebf0 tests: Make apply-events tests easier to debug.
It's fairly difficult to debug tests that use
EventsRegisterTest.do_test, and when they fail on
Travis, it's particularly challengning.  Now we make
the main diff less noisy, and we also include
the events that were applied.
2017-10-06 13:56:36 -07:00
derAnfaenger 747c283180 lint: Replace `pk` with `id` where they are equivalent. 2017-10-06 12:52:12 -07:00
Tim Abbott 0c44310009 tests: Don't use caching session engine in tests.
Because we run tests in parallel, the caching session engine can lead
to nondeterministic failures.
2017-10-06 12:22:20 -07:00
Steve Howell a331b4f64d Optimize query_all_subs_by_stream().
Using lightweight objects will speed up adding new users
to realms.

We also sort the query results, which lets us itertools.groupby
to more efficiently build the data structure.

Profiling on a large data set shows about a 25x speedup for this
function, and before the optimization, this function accounts
for most of the time spend in bulk_add_subscriptions.

There's a lot less memory to allocate.  I didn't measure
the memory difference.

When we test-deployed this to chat.zulip.org, we got about a 6x
speedup.
2017-10-06 11:03:44 -07:00
Steve Howell f5ddc40d14 Have get_peer_user_ids_for_stream_change() use user_ids. 2017-10-06 11:03:44 -07:00
Tim Abbott fe6a04cbdb Revert "locale: Fix incorrect locale name for zh-Hans."
This reverts commit ba8dc62132.

As best I can tell, the old configuration was correct for what Django
wanted.  Further testing is required, but this at least brings
.tx/config to match the actual filenames; I think our Chinese
translations have been broken until now.
2017-10-06 10:09:08 -07:00
Tim Abbott 0f2e4a55c0 soft deactivation: Shorten management command name.
This command is really for soft deactivation; there's just an undo
feature.
2017-10-06 08:48:43 -07:00
Tim Abbott ca59b28fc1 import: Fix positional argument parsing. 2017-10-06 08:45:22 -07:00
Tim Abbott bfedb82097 import: Fix argparse usage message breakage. 2017-10-06 08:41:01 -07:00
Tim Abbott 1a9adbbe70 test_signup: Fix test for login page redirect.
I forgot to update this in 43bba27e5d.
2017-10-05 23:36:33 -07:00
Tim Abbott 313083484a test_i18n: Remove zh-hans tests for now. 2017-10-05 23:33:12 -07:00
Tim Abbott 43bba27e5d auth: Fix redirect loop in development environment.
Since the REALMS_HAVE_SUBDOMAINS migration in development, we've had
scattered reports of users who found trying to open 127.0.0.1:9991
resulting in a redirect loop between zulipdev.com:9991,
zulipdev.com:9991/devlogin, and zulipdev.com:9991/devlogin/, and back
to zulipdev.com:9991.

We fix this temporarily through a small cleanup, which is to have that
last step in the loop send the user to the subdomain where they're
actually logged in, zulip.zulipdev.com:9991.

There's more to be done before this system will make sense, though.
2017-10-05 23:21:29 -07:00
Tim Abbott 46485322eb middleware: Remove logic for redirecting to zulipdev.com domains.
We originally wrote this because when testing subdomains, you wanted
to be sure you were actually testing subdomains.  Now that subdomains
is the default, doesn't seem to actually be a good reason why we
should need this.
2017-10-05 23:21:02 -07:00
Umair Khan 6fbf41bdbc queue: Add test for signups retry logic. 2017-10-05 23:14:19 -07:00
Umair Khan 435fe40199 worker: Retry signups queue event on 400. 2017-10-05 23:14:19 -07:00
Umair Khan 19e2551e82 mypy: Change type to Dict for SignupWorker. 2017-10-05 23:14:19 -07:00
Tim Abbott f11832536c event_queue: Fix a now-incorrect mypy annotation.
It's not clear tagging things like this as an Iterable is a good idea,
when it's in fact definitely a list.
2017-10-05 23:11:16 -07:00