Commit Graph

21865 Commits

Author SHA1 Message Date
Steve Howell 75db4f5187 Avoid O(N) query pitfall for /users endpoint.
We now use a `.values` query to get just the fields we need
in order to fulfill '/json/users' requests.

The main benefit is that we don't do O(N) queries for bot
owners, but we also have less data on UserProfile to process.
2017-10-09 14:08:07 -07:00
Steve Howell f87159ad98 Flush standard output in match_states(). 2017-10-09 12:23:49 -07:00
Eeshan Garg 0ca1224b3e integrations: Render xkcd bot's documentation. 2017-10-09 11:40:44 -07:00
Eeshan Garg 1153288681 markdown: Add macro linking to our Bots Guide tutorials.
This macro is for when we want to describe how to run a bot on
a running Zulip server in our bots' docs.
2017-10-09 11:40:44 -07:00
Eeshan Garg 48be9e3e44 packages: Upgrade Zulip API packages (0.3.4 -> 0.3.5). 2017-10-09 11:40:36 -07:00
Harshit Bansal 6f4d24d851 reactions.js: Fix the conditional check for sending reaction request.
Request for adding an reaction only if there is a default emoji or
an active realm emoji with that name while request for removing a
reaction should be sent only if there is a default emoji or a realm
emoji(may be active or deactivated) with that name. Earlier we were
not including deactivated realm emojis while deciding whether a
request for removing a reaction should be sent or not which was
causing requests for the removal of reactions with deactivated realm
emojis not to be sent to the backend.

Fixes: #6007.
2017-10-09 11:31:21 -07:00
Harshit Bansal 3acaa79336 reactions.py: Don't check for valid emoji name while removing reaction.
On receiving a request for deleting a reaction, just check if such
a reaction exists or not. If it exists then just delete the reaction
otherwise send an error message that such a reaction doesn't exist.
It doesn't make sense to check whether an emoji name is valid or not.
2017-10-09 17:54:37 +00:00
Eeshan Garg 71eee35bce webhooks: Add a Google Code-in integration. 2017-10-09 09:04:39 -07:00
Tim Abbott 9cf26db6db test_management_commands: Fix send_webhook_fixture_message test.
I unfortunately didn't remember this could have a test.
2017-10-08 21:20:35 -07:00
Steve Howell a67b07bfc4 Avoid shadowing stream variables.
This helps with mypy typings in an upcoming commit.
2017-10-08 20:18:34 -07:00
Steve Howell 2e43562832 Extract Stream.num_subscribers_for_stream_id.
This allows us to count subscribers without a Stream object.
2017-10-08 20:18:34 -07:00
Steve Howell 12e65eb21c Use stream ids in various tests.
This commit prepares us to introduce a StreamLite class. For
these tests, we don't care about the actual contents of the
Stream, just the right stream is there.
2017-10-08 20:18:34 -07:00
Steve Howell c1d7fc6e80 Only require stream_id in private_stream_user_ids(). 2017-10-08 20:18:34 -07:00
Steve Howell 7dbea8a2bf Only require stream_id in subscribed_to_stream().
Since subscribed_to_stream is only doing an id lookup
on the Stream model to find out if a user is subscribed to
a stream, there's no reason to require a full Stream object.

It's currently the case that all callers do have full Stream
objects handy to pass in to this function, but it's still a
good practice to have functions only ask for objects that they
need.
2017-10-08 20:18:34 -07:00
Tim Abbott 49274a060c send_webhook_fixture_message: Fix to work with subdomains.
We apparently failued to update this tool when we switched the
development environment to require a subdomain.
2017-10-08 20:14:14 -07:00
Tim Abbott 416f34c80e setup-production: Try to fix Travis failures by blocking mongodb repo.
It appears the mongodb repo is not accessible by Travis CI right now.
This is sadly our problem, because Travis puts a bunch of crap in
their apt `sources.list` file, so `apt-get update` starts failing.
2017-10-08 20:06:42 -07:00
Tim Abbott 88bb6c6cad uploads: Set a BOTO_CONFIG path to override broken GCE configuration.
The comment is pretty self-explanatory.  The fact that Google Compute
Engine has this problem does not impress confidence about their
product, but hopefully this is the only really dumb thing they do.

Fixes #4839.
2017-10-08 19:46:57 -07:00
Greg Price 0691724836 passwords: Set default zxcvbn threshold to 10k guesses.
See the discussion in the revised docs for background and motivation,
and an explanation of why this value.
2017-10-08 15:48:44 -07:00
Greg Price a116303604 passwords: Express the quality threshold as guesses required.
The original "quality score" was invented purely for populating
our password-strength progress bar, and isn't expressed in terms
that are particularly meaningful.  For configuration and the core
accept/reject logic, it's better to use units that are readily
understood.  Switch to those.

I considered using "bits of entropy", defined loosely as the log
of this number, but both the zxcvbn paper and the linked CACM
article (which I recommend!) are written in terms of the number
of guesses.  And reading (most of) those two papers made me
less happy about referring to "entropy" in our terminology.
I already knew that notion was a little fuzzy if looked at
too closely, and I gained a better appreciation of how it's
contributed to confusion in discussing password policies and
to adoption of perverse policies that favor "Password1!" over
"derived unusual ravioli raft".  So, "guesses" it is.

And although the log is handy for some analysis purposes
(certainly for a graph like those in the zxcvbn paper), it adds
a layer of abstraction, and I think makes it harder to think
clearly about attacks, especially in the online setting.  So
just use the actual number, and if someone wants to set a
gigantic value, they will have the pleasure of seeing just
how many digits are involved.

(Thanks to @YJDave for a prototype that the code changes in this
 commit are based on.)
2017-10-08 15:48:44 -07:00
Greg Price 11e767f5b1 passwords: Revise our password-quality logic for clarity.
This should be a pure refactor: we're just removing or tightening
some obsolete comments, and rearranging some logic to make it
easier to follow.
2017-10-08 15:48:44 -07:00
Tim Abbott 1ceaedb2c5 docs: Remove html_unescape.py.
This was just for 1-time use exported the original Zulip documentation.
2017-10-08 15:41:41 -07:00
rht fbd24669a4 Cleanup: Remove shebang and executable flag from these files. 2017-10-08 15:38:15 -07:00
rht de30400fc5 pg_backup_and_purge.py: Remove .py extension. 2017-10-08 15:32:43 -07:00
Tim Abbott ec080aed6b mypy: Workaround lxml annotations being busted. 2017-10-08 12:38:20 -07:00
Tim Abbott d215ea1e37 actions: Rename all_subs_by_stream to all_subscribers_by_stream.
The previous name sounded a bit too much like they were subcription
objects.
2017-10-08 12:33:53 -07:00
Steve Howell 3e6bfe1b23 Use user_ids, not emails, for bulk stream operations.
We now return user_ids for subscribers to streams in add-stream
events.  This allows us to eliminate the UserLite class for
both bulk adds and bulk removes.  It also simplifies some JS
code that already wanted to use user_ids, not emails.

Fixes #6898
2017-10-08 12:31:12 -07:00
Harshit Bansal 3c434f0d86 notifications: Switch to use `make_links_absolute()` from lxml library.
Instead of using custom regexes for converting relative URLs to
absolute URLs switch to using `make_links_absolute()` function
from lxml library.
2017-10-08 12:15:30 -07:00
Steve Howell 810532641e activity.js: Extract get_filtered_and_sorted_user_ids().
This function was extracted from build_user_sidebar().  We
also slightly streamlined it to not unnecessarily call
filter() when the filter text was blank. This extraction
also eliminated the need for us to have the two-line
filter_and_sort() function.

Also, we get to 100% coverage in this commit.
2017-10-08 12:01:59 -07:00
Steve Howell 6888255153 Add people.get_realm_user_ids().
This saves us from doing an unnecessary map() in
activity.js.
2017-10-08 12:01:59 -07:00
Steve Howell 943eefb7c0 activity.js: Extract get_filter_text().
We were using slightly different logic to get the filter
text in various places.  Now we're consistent, and it's
easier to test the edge-case behavior.
2017-10-08 12:01:59 -07:00
Steve Howell 3ef2df363b activity.js: Consolidate initialization code.
We now intialize user-list-filter within activity.initialize(),
which gives us more control to set the module variable
`meta.$user_list_filter` before we build the user sidebar,
while setting up its handlers after we build the sidebar.
2017-10-08 12:01:59 -07:00
Steve Howell 662b5ffbba activity.js: Remove inaccurate comment.
The comment removed here doesn't make sense, since meta
is not exported.
2017-10-08 12:01:59 -07:00
Steve Howell 86d372f9ce lint: Prevent "userid" in code.
We should always use user_id for consistency sake.
2017-10-07 12:16:45 -07:00
Steve Howell 10a30bece1 Rename presence_idle_userids -> presence_idle_user_ids. 2017-10-07 12:16:45 -07:00
Steve Howell e3ee0245c4 Change userid -> user_id in people.js. 2017-10-07 12:16:45 -07:00
Steve Howell fbaef43ac3 Rename bot_owner_userids -> bot_owner_user_ids. 2017-10-07 12:16:45 -07:00
Greg Price aa4104a5af logging: Add option to show the PID in each log message. 2017-10-06 19:21:40 -07:00
Greg Price 9cf44a77da lint: Narrow the space-around-% patterns, remove exclusions.
This lint rule has bitten me a couple of times in working on logging.
These regex rules will inevitably be heuristic, but we can make it a bit more
specific so that the heuristic mainly means it could occasionally miss
something, rather than get in the way with an obviously wrong complaint.
2017-10-06 19:21:40 -07:00
Brock Whittaker 4d33e66496 /terms/: Convert "terms" page to be styled like why/for pages.
This styles it to be like the why/for landing pages and have the new
small hero.
2017-10-06 17:01:34 -07:00
Brock Whittaker 887b4d9fa8 /why-zulip/: Change "why" page to use smaller hero. 2017-10-06 17:01:34 -07:00
Brock Whittaker 8d3ebd8b47 /for/*: Add ".small" version of why-page hero.
This adds a small version that has 130px less padding than the
original header.
2017-10-06 17:01:34 -07:00
Brock Whittaker 057f9bafb0 Restyle privacy page to look like /for/ and "why" pages.
This restyles the privacy page from an older style to a new
updated style with the mini-hero and naturally readable width
text.
2017-10-06 17:01:34 -07:00
Harshit Bansal 7b88e3eea7 tests: Add a test suite for verifying the mobile push notifs content.
This test suite works by using the expected_output and new text_output
fields in the bugdown test cases to verify that each syntax is
correctly translated by this new function.

Some of these translations, like strikethrough, are kinda poor; but
this framework should make it easy to iterate on the formatting.

Fixes: #6720.
2017-10-06 16:47:27 -07:00
Harshit Bansal 5a6584890d push_notifications: Start using `get_mobile_push_content()` function. 2017-10-06 16:47:25 -07:00
Harshit Bansal 28628eeaeb push_notifications: Add `truncate_content()` function.
This function truncates the textual content at correct length.
(It will be updated later to handle corner cases of unicode
combining characters and tags when we start supporting them.)
2017-10-06 16:44:19 -07:00
Harshit Bansal b5a1aacfb3 push_notifications: Add `get_mobile_push_content()` function.
Given the rendered content of a message, this function strips
all the markup replacing emojis with their corresponding unicode
representation.
2017-10-06 16:44:18 -07:00
Harshit Bansal 2b7dd8e437 requirements: Add 'lxml' as a dependency required for parsing HTML.
We need to parse rendered HTML content of messages while preparing
content for mobile push notifications and for doing so we need to
use lxml's HTML parser.
2017-10-06 16:37:17 -07:00
Tim Abbott 47c5aae5b2 log2zulip: Enforce using python 3 in cron job.
We aren't guaranteed to have the Zulip dependencies installed on
Python 2.
2017-10-06 16:37:17 -07:00
Tim Abbott a197b82eb5 requirements: Upgrade mypy to version 0.530. 2017-10-06 15:55:11 -07:00
Tim Abbott 234f7960a1 tests: Add @slow decorator to various methods needing it.
Also, fix the test_message_editing test having way too many cases.

Fixes #1478.
2017-10-06 15:45:04 -07:00