Commit Graph

23046 Commits

Author SHA1 Message Date
Greg Price d6cfa56bc1 test_signup: Test that cloning a system bot's email is not allowed.
Just now this is largely redundant with `test_signup_already_active`;
but very soon when we allow reusing an email across realms, the logic
will diverge.
2017-11-28 16:23:10 -08:00
Greg Price a753f49e6a populate_db: Add a "lear" realm for testing cross-realm email sharing.
We don't yet actually allow sharing emails across realms, but we
will soon.  This lets us start testing some pieces of those changes.
2017-11-28 16:23:10 -08:00
Greg Price 5f38b374b6 register: Activate existing mirror dummy only if on same realm.
The one thing this bit of logic is used for is to decide whether
there's an existing user which is a mirror dummy that we should
activate.  This change causes us to ignore such an existing user if
it's on some other realm, and go straight into `do_create_user`.
2017-11-28 16:23:10 -08:00
Greg Price eb8b05b2b5 test_email_mirror: Remove a brittle bit of logic.
This code appears to exist to cover a few extra lines in
zerver/lib/digest.py.  But it's rather brittle, tucked as it is into
the middle of a different test's loop, and with the upcoming
introduction of the `lear` realm in testing, this test code itself
loses coverage.

For now, rather than fix this test code up just delete it; we don't
have 100% coverage on `zerver/lib/digest.py`, while we do on this test
file, so that avoids breaking coverage in CI.  As a followup, we
should add back some logic like this but in a more robust way,
probably as its own separate test method.
2017-11-28 16:23:10 -08:00
Greg Price 453585abb1 test_messages: Clean up some error spew by updating a test.
This test would produce a bunch of log messages with tracebacks,
complaining that `welcome-bot@zulip.com` tried to send cross-realm PMs
and can't.  The issue is that the test overrides
`settings.CROSS_REALM_BOT_EMAILS`, and hasn't kept up with additions
to the normal value for that setting.  Update it so that welcome-bot
is permitted as usual.
2017-11-28 16:23:10 -08:00
Greg Price f632692188 lint: Fix use of re.match in judging long lines.
The `re.match` function in the Python stdlib is a trap for the unwary,
with surprising and asymmetrical semantics; we should probably add a
lint rule to ban it entirely.  The docstring says:

> Try to apply the pattern at the start of the string, [...]

In other words, it effectively adds a `^` at the start (or `\A`, where
the distinction matters.)  It's bad enough that this differs from what
grep, sed, perl, less, and every other tool I can think of do when
looking for matches to a regex; on top of that, it treats the
beginning of the string differently from the end, for no obvious
reason.  The function that does what the rest of the world understands
by "match against this regex" is `re.search`.

In this case, it's unlikely that anyone intended for comments with
URLs, or `api_url` references, to miss out on their respective
exceptions to the long-line rule if they happen to start after the
first column.  So fix those rules by just switching to `re.search`
with the same pattern.

I think Markdown URL references may have to start at the beginning of
the line, so I've left a `^` there to preserve -- but now make
explicit -- the `re.match` behavior.
2017-11-28 16:23:10 -08:00
Greg Price b6cc21b438 debug: Add facility to dump tracemalloc snapshots.
Originally this used signals, namely SIGRTMIN.  But in prod, the
signal handler never fired; I debugged fruitlessly for a while, and
suspect uwsgi was foiling it in a mysterious way (which is kind of
the only way uwsgi does anything.)

So, we listen on a socket.  Bit more code, and a bit trickier to
invoke, but it works.

This was developed for the investigation of memory-bloating on
chat.zulip.org that led to a331b4f64 "Optimize query_all_subs_by_stream()".

For usage instructions, see docstring.
2017-11-28 15:52:07 -08:00
derAnfaenger 6dd639454e integrations: Add topic parameter for Appfollow.
Tweaked by tabbott to document the integration.
2017-11-28 15:49:19 -08:00
Rishi Gupta 191f8f1b39 left sidebar: Change Home to All messages.
We made this change because users often unnecessarily click "Home"
first in their use of Zulip, because it seems appealing.  While "All
messages" isn't quite precise (it doesn't include muted streams), it
does describe relatively simply the interleaved view that this
represents.

This commit leaves everything as "home" in the code, and only changes
user-visible strings and docs. Changing the code will be a big project;
there are hundreds of relevant occurrences in variable names, etc.

Further, we'll probably want to convert those various variable names
in different ways.

Tweaked by tabbott to extend the commit message and update a few comments.
2017-11-28 15:42:58 -08:00
Umair Khan 1acdfef13c two_factor: Disable prefix in OurAuthenticationForm.
In two factor authentication every step adds a unique prefix to the fields,
due to this the name of the form fields differs from the HTML fields. If
we do not do this we will have to change the name in the HTML, which
will cause the change in tests.
2017-11-28 15:27:44 -08:00
Umair Khan 274bba82b9 two_factor: Add configuration and URLs.
This adds django-two-factor to the project, but held behind
settings.TWO_FACTOR_AUTHENTICATION_ENABLED, so that this has no effect
by default.
2017-11-28 15:21:40 -08:00
Tim Abbott 4f5a5a8547 auth: Extract add_dev_login_context.
This simple refactor cleans up the code for the dev_auth_enabled code
path to be a bit less deeply in the login_page() logic.
2017-11-28 15:17:33 -08:00
Tim Abbott b42ce65090 requirements: Add dependencies for two-factor auth.
We're not yet ready to add 2FA to Zulip yet, but we've determined we'd
like to work from these libraries.

I'm not bothering to bump PROVISION_VERSION for this, since we're
likely to do something else that bumps it soon, and we're not merging
anything that uses these new libraries.
2017-11-28 15:13:38 -08:00
Tim Abbott d5bb45db60 webhooks/gci: Fix missing mypy annotation. 2017-11-28 14:08:44 -08:00
Sampriti Panda bea653fabc webhooks/gci: Improve message formatting.
* Use Student Name instead of Task Name in subject.
* Use Task Instance URL instead of Task Definition URL (and workaround
  for a bug in the API).
2017-11-28 13:53:42 -08:00
Tommy Ip 2d7536bf05 Document the add_mentor.py tool. 2017-11-28 13:26:59 -08:00
Tommy Ip ba1a32ca74 Create script to add and remove mentor's ssh key from DO droplets. 2017-11-28 13:26:59 -08:00
Tommy Ip 6e22847548 refactor: new message content -> compose textarea. 2017-11-28 12:53:40 -08:00
Cynthia Lin e92a8ac8ee subs: Properly focus on Stream name box while creating a new stream.
Fixes #7473.
2017-11-28 10:07:11 -08:00
Garvit b9826e9067 Add filters for toMarkdown to paste data to compose-box.
This fixes bugs with pasting headings and italic styling.

Fixes #7485.
2017-11-28 09:56:48 -08:00
Tim Abbott cb8f2cf484 git cheat sheet: Be a bit less agressively anti-git-pull.
Now that we configure pull.rebase, it's less necessary.
2017-11-28 09:15:13 -08:00
Rohitt Vashishtha c073af0ddc git-docs: Highlight Zulip's git scripts in overview. 2017-11-28 09:07:25 -08:00
Rohitt Vashishtha 106cc71c19 docs: Add external documentation for markdown subsystem.
Tweaked by tabbott to move the content earlier in the doc.
2017-11-28 09:07:17 -08:00
Rohitt Vashishtha a2d1ebc091 git-docs: Refine git-cheat-sheet with more examples. 2017-11-28 09:02:50 -08:00
Vishnu Ks 87a412a992 droplets: Fix broken link for requesting droplets. 2017-11-28 08:53:52 -08:00
Vishnu Ks 73ee9a4ad8 droplets: Update info on recreating the droplets. 2017-11-28 08:53:52 -08:00
Vishnu Ks c37193568f droplets: Update baseimage snapshot id.
The new snapshot has python-zulip-env as well
as VIM and mosh installed.
2017-11-28 08:53:52 -08:00
Tommy Ip 29e9d16ac7 refactor: s/upload-bar/compose-upload-bar. 2017-11-27 21:35:14 -08:00
Tommy Ip 6a694418b2 refactor: s/error-msg/compose-error-msg. 2017-11-27 21:35:14 -08:00
Tommy Ip b9c2f479d3 refactor: s/send-status-close/compose-send-status-close. 2017-11-27 21:35:14 -08:00
Tommy Ip c0c58f9761 refactor: s/send-status/compose-send-status. 2017-11-27 21:35:14 -08:00
Tommy Ip 19b518c801 refactor: Extract tests for upload mechanics.
This temporarily removes the tests for clear_out_file_list since
fixing that test proved to be difficult.
2017-11-27 21:34:55 -08:00
Tommy Ip ddaff4cd2a refactor: Extract upload mechanics to new JS module.
Tweaked by tabbott to move changes from the next commit that are
required for this to pass tests into this commit.

Note that this exports a few items that were not previously exported.
2017-11-27 21:31:51 -08:00
Tim Abbott 0445322625 compose: Export the major upload-related methods.
This is a preparatory commit to moving these to their own module.
2017-11-27 21:31:24 -08:00
Tim Abbott df58f0f7ae compose: Export some upload-related variables. 2017-11-27 21:31:23 -08:00
Robert Hönig 1e66bd1c02 tests: Add newlines to test_service_bot_system.test_internal_endpoint(). 2017-11-27 21:05:34 -08:00
derAnfaenger c8a5ae753c embedded bots: Consistently use 'storage' instead of 'state.' 2017-11-27 21:05:34 -08:00
Tim Abbott 9645c8b31f integrations: Remove disabled googlesearch integration.
This was moved to another part of the python-zulip-api codebase
because it was unfinished.
2017-11-27 20:50:37 -08:00
Eeshan Garg c45517f544 python-zulip-api: Upgrade to PyPI package release 0.3.8.
There's one migration required by this release:

* queue_processors: Stop passing state_handler to handle_message.

  state_handler is now a property of bot_handler and thus, does
  not need to be passed to bot_handler.handle_message().

  The commit responsible is:
  2a74ad11c5
2017-11-27 20:31:37 -08:00
rht e538f4dd44 zerver/views: Use Python 3 syntax for typing.
Edited by tabbott to remove state.py and streams.py, because of
problems with the original PR's changes, and wrap some long lines.
2017-11-27 17:10:39 -08:00
rht 92888a0cde zproject: Use Python 3 syntax for typing. 2017-11-27 17:01:18 -08:00
Rishi Gupta 83149a953a copy paste: Turn off toMarkdown in production.
At least until some of the TODOs at the bottom of
node_tests/copy_and_paste.js are done.
2017-11-27 16:44:16 -08:00
Rishi Gupta 3fcfcd2a55 copy paste: Add toMarkdown test for `1. text`. 2017-11-27 16:44:16 -08:00
Tim Abbott bd8196c3a5 lint: Fix too-long line in auth.py. 2017-11-27 15:28:46 -08:00
Rohitt Vashishtha 8082780853 typeahead: Update @mention to not autocomplete "@ " (@space).
Tweaked by tabbott to also cover "@** ".

Fixes #7533.
2017-11-27 15:15:56 -08:00
Vishnu Ks d4ee3023b2 registration: Require an explicit realm on PreregistrationUser.
This completes the last commit's work to fix CVE-2017-0910, applying
to any invite links already created before the fix was deployed.  With
this change, all new-user registrations must match an explicit realm
in the PreregistrationUser row, except when creating a new realm.

[greg: rewrote commit message]
2017-11-27 14:59:34 -08:00
Vishnu Ks 985768b2fd registration: Check realm against PreregistrationUser realm.
We would allow a user with a valid invitation for one realm to use it
on a different realm instead.  On a server with multiple realms, an
authorized user of one realm could use this (by sending invites to
other email addresses they control) to create accounts on other
realms. (CVE-2017-0910)

With this commit, when sending an invitation, we record the inviting
user's realm on the PreregistrationUser row; and when registering a
user, we check that the PregistrationUser realm matches the realm the
user is trying to register on.  This resolves CVE-2017-0910 for
newly-sent invitations; the next commit completes the fix.

[greg: rewrote commit message]
2017-11-27 14:58:26 -08:00
Tim Abbott 8b935f4e99 settings: Add setting for SYSTEM_BOT_REALM.
This fixes some subtle JavaScript exceptions we've been getting in
zulipchat.com, caused by the system bot realm there not being "zulip"
interacting with get_cross_realm_users.
2017-11-27 14:46:07 -08:00
Tim Abbott 25fd4c5508 bulk_get_users: Edit the cache keys to make them more unique.
While at this point I was to rewrite this function, this at least
plugs the issues for now.
2017-11-27 14:41:31 -08:00
Tim Abbott 339b67f7af get_cross_realm_dicts: Harden against caching bug in last commit.
This should help protect us from future issues with the way that
`bulk_get_users` does caching.

It's likely that we'll want to further restructure `bulk_get_users` to
not have this base_query code path altogether (since it's kinda
buggy), but I'm going to defer that for a time when we have another
user.
2017-11-27 14:35:50 -08:00