Commit Graph

34195 Commits

Author SHA1 Message Date
Mateusz Mandera 8d987ba5ae auth: Use tokens, with data stored in redis, for log_into_subdomain.
The desktop otp flow (to be added in next commits) will want to generate
one-time tokens for the app that will allow it to obtain an
authenticated session. log_into_subdomain will be the endpoint to pass
the one-time token to. Currently it uses signed data as its input
"tokens", which is not compatible with the otp flow, which requires
simpler (and fixed-length) token. Thus the correct scheme to use is to
store the authenticated data in redis and return a token tied to the
data, which should be passed to the log_into_subdomain endpoint.

In this commit, we replace the "pass signed data around" scheme with the
redis scheme, because there's no point having both.
2020-01-26 21:32:44 -08:00
Hashir Sarwar 91aff4eec3 invite_subscription: Add `#` to the stream names.
Written the stream names as e.g. #announce, to match
how we display them elsewhere.
2020-01-26 21:20:11 -08:00
Tim Abbott 9ac3e1099c analytics: Remove last_modified field from FillState.
This field wasn't used for anything, and I think it has very limited
use for debugging, since fundamentally, it'll almost always have a
value within the hour of the actual timestamp in FillState, and any
more fine-grained logging we might want would be available in the
analytics job's own logs.

The proximal reason to remove it is that apparently Django's
model_to_dict doesn't support auto_now fields, and that caused some
trouble when working on adding more complete import/export support for
analytics data.
2020-01-26 20:38:26 -08:00
Tim Abbott d7ba77b83b search: Simplify CSS for search_icon.
This is preparatory refactoring for replacing the top navbar UI; in
the new version, the search_icon may not be an <a> tag.
2020-01-25 23:55:48 -08:00
Tim Abbott d356622594 docs: Add link from LDAP docs to invitation docs.
This addresses confusion we had with some organizations where they
were surprised that with only LDAP enabled, the "invite more users"
feature was available.

Fixes #11685.
2020-01-25 23:41:19 -08:00
Abhishek-Balaji 434e8d3104 home: Extract compute_show_invites_and_add_streams.
This extracts a function for computing show_invites and
show_add_streams, for better readability and testability.

This commit was substantially cleaned up by tabbott.
2020-01-25 23:41:08 -08:00
Rohitt Vashishtha 39420d6adb night_mode: Render the message instead of showing raw `/night` text. 2020-01-25 22:56:45 -08:00
Vishnu KS 97a25657a5 tests: Set class name of video call test to TestVideoCall.
The previous class name TestFeedbackBot was probably
leftover from copy paste.
2020-01-25 22:54:59 -08:00
Vishnu KS 05b4610381 bots: Remove feedback cross realm bot.
This completes the remaining pieces of removing this missed in
d70e799466 (mostly in tests).
2020-01-25 22:54:44 -08:00
Tim Abbott 122e11c678 slack import: Fix handling of messages sent by user U00. 2020-01-25 22:47:49 -08:00
Tim Abbott d70e799466 bots: Remove FEEDBACK_BOT implementation.
This legacy cross-realm bot hasn't been used in several years, as far
as I know.  If we wanted to re-introduce it, I'd want to implement it
as an embedded bot using those common APIs, rather than the totally
custom hacky code used for it that involves unnecessary queue workers
and similar details.

Fixes #13533.
2020-01-25 22:41:39 -08:00
Vishnu KS a4dcdd9c7b emails: Set alt attribute to empty for leading images.
The alt text of the leading images were displayed as preview
content in inbox by email clients like gmail. Since the leading
images were used mostly for decoration this made the preview
content gibberish. It's fine to set the alt attributes to empty
from accessibility point of view since the old alt attributes
did't added any meaningful information.
2020-01-24 13:32:54 -08:00
rht 7c9954afc6 install: Reduce the RAM limit to accommodate some definition of "2GB".
See also the discussion at
https://chat.zulip.org/#narrow/stream/3-backend/topic/2GB.20ram.20gotcha.
2020-01-24 12:35:37 -08:00
arpit551 b23a5431cd analytics: Add realm argument to analytics.
This changeset is prepartory work for doing something reasonable with
analytics data during the zulip -> zulip data import process (and
potentially e.g. slack -> Zulip as well).

To support that, we need to make it possible to do our analytics
calculations for a single realm.

We do this while maintaining backwards compatibility and avoiding
massive duplicated code by adding an optional `realm` argument to the
entrypoints to the analytics system, especially process_count_stat.

More work involving restructuring FillState will be required for this
to be actually usable for its intented purpose, but this commit is a
nice checkpoint along the way.

Tweaked by tabbott to adjust comments and disable InstallationCount
updates when a realm argument is specified.
2020-01-23 17:36:13 -08:00
Mateusz Mandera 3f99985df3 auth: Require only subdomain and email in log_into_subdomain.
Tweaked by tabbott to add a comment explaining the reasoning.
2020-01-23 16:42:35 -08:00
Mateusz Mandera 90fe1bda02 auth: Extract create_response_for_otp_flow function. 2020-01-23 16:37:55 -08:00
Mateusz Mandera af2c4a9735 redis: Extract put_dict_in_redis and get_dict_from_redis helpers. 2020-01-23 16:24:07 -08:00
Mateusz Mandera eafdefc5c9 auth: Extract finish_mobile_flow function. 2020-01-23 16:24:07 -08:00
Mateusz Mandera bce50ee652 auth: Use authenticate_remote_user in remote_user_jwt.
authenticate_remote_user already takes care of calling the authenticate
with the dummy backend. Also, return_data is not used and catching
DoesNotExist exception is not needed, as the dummy backend just returns
None if user isn't found.
2020-01-23 16:24:07 -08:00
Mateusz Mandera f00a852697 auth: Extract register_remote_user function. 2020-01-23 16:24:07 -08:00
Mateusz Mandera 27b9eafcac social_auth: Set is_signup=False if the user is already signed up.
Because of how login_or_register_remote_user code is structured, this
doesn't change how the flow will go, but it's not a clean use of
login_or_register_remote_user to call it with is_signup=True if sign up
shouldn't actually happen - and may be fragile when refactoring
login_or_register_remote_user.
2020-01-23 16:24:07 -08:00
Mateusz Mandera 2f38bc669f auth: Set request._email to user.delivery_email instead of user.email.
In other places where we set request._email, we set it to the
delivery_email, as that's more informative in orgs with hidden email
settings, where user.email will be useless.
2020-01-23 16:24:07 -08:00
orientor 030c07184a settings: Fix buggy emoji format loading spinner.
When a user clicked the current emoji format in "display settings",
we'd show an infinite loading spinner (basically as a side effect of
trying to tell the server to change the emoji format to what it
already was).

Fix this by aborting early if the emoji format is already the option
that the user clicked.

Fixes #13684.
2020-01-23 16:20:57 -08:00
Steve Howell 7d0082f5c2 topic history: Avoid duplicate server fetches.
We now only go the server if both of these
conditions are true:

    - our message data seems incomplete for
      the stream
    - we haven't already fetched history
2020-01-23 13:28:52 -08:00
Steve Howell e812dd60d6 Extract topic_data.is_complete_for_stream_id().
This function will make more sense when we start
tracking api calls that retrieve topic history.

The unit tests here are kinda duplicating what we
have in the stream_data tests.  If we move the
function out of stream_data, we can kill off the
tests there, but for now I think a bit of duplicate
testing is fine here.
2020-01-23 13:28:52 -08:00
Steve Howell 779a9cea22 topic_data: Use IntDict for stream_dict.
All the callers seem to have integer stream_ids
already, either from the message object or
some sub object.

We also use clear() inside the test-only reset()
method.
2020-01-23 13:28:52 -08:00
Steve Howell 055b13a7a8 minor: Update code comments in topic_data.
The main Dict here used to point to an array,
but that changed a few years ago.
2020-01-23 13:28:52 -08:00
rht 6f5cbed5f1 prod install: Ignore reading VERSION_CODENAME if not available. 2020-01-22 23:19:18 -08:00
rht c2dcaf48d8 prod install: Add the CentOS version of SUPERVISOR_CONF_DIR. 2020-01-22 23:19:18 -08:00
Tim Abbott a3a662c7d2 message_list_view: Fix handling of links to deleted streams.
Previously, links to deleted streams would be incorrectly rendered as
stream's name).

Fixes an issue that was reported where after deleting the "general"
stream, the welcome turtle messages might appear as links to
2020-01-22 22:42:54 -08:00
Tim Abbott e052ec58db slack import: Improve error messages around invalid tokens.
This updates our error handling of invalid Slack API tokens (and other
networking error handling) to mostly make sense:
* A token that doesn't start with `xoxp-` gives an extended error early.
* An AssertionError for the codebase is correctly declared as such.
* We check for token shape errors before querying the Slack API.

We could still do useful work to raise custom exception classes here.

Thanks to @stavrospat for raising this issue.
2020-01-22 14:48:32 -08:00
Steve Howell f24de074d4 node tests: Get 100% coverage on topic_list_data. 2020-01-22 14:31:33 -08:00
Steve Howell fa82d12525 topic list: Extract topic_list_data.js.
This is mostly for tactical reasons.  It's hard to
get 100% test coverage on topic_list.js, but it
should be easy to get 100% test coverage on this
very important function.

I considered just moving this code into topic_data.js,
but it just didn't feel quite right.  I feel like
this is a pretty core piece of code that's nice
to be by itself and not be near other complicated
code that does stuff like build widgets or talk
to servers.  (And, again, it's not just the actual
code here, which is pretty small, it's the unit
tests, which are inherently verbose to exercise
all the edge cases.)
2020-01-22 14:31:33 -08:00
Steve Howell 588f34e02f node tests: Remove complicated topic_list test.
This test mostly tests how we glue everything
together, but I want to change that in an upcoming
commit.

Also, the data stuff that it tests is now better
covered by the test recent tests I added.
2020-01-22 14:31:33 -08:00
Steve Howell a7a47fc730 node tests: Test muting-related changes.
This test exercises some code from Tim's recent
patch to exclude muted topics from the topic
list.
2020-01-22 14:31:33 -08:00
Steve Howell 6d4c16c8d7 tests: Add coverage for topic_list.get_list_info(). 2020-01-22 14:31:33 -08:00
Steve Howell 0f368b3373 topic_list: Fix minor bug with "more topics".
There was an edge case with the old
code when you had exactly between 6 and 8
topics and all in cache, with a couple of
the topics being unread.

We would show "more topics" when you were
actually seeing all your possible topics.

To test this:
    - create 7 topics on Venice
    - as Iago, narrow to any of the Venice
      topics
    - as Aaron, send unreads to 3 or 4
      of the other topics

Eventually Iago will have all possible
topics in the sidebar.  On master we'll
show "more topics", whereas after this commit
we correctly avoid that.

It's a pretty harmless bug, since it just
leads to a useless zoom-in.

I have always felt we should zoom-in
regardless of how many topics you have,
just for consistency sake, but I also
understand the rationale behind our
current intentions.
2020-01-22 14:31:33 -08:00
Steve Howell 27bcdc567f topic_list: Extract get_list_info.
This is basically trying to confine the
rendering logic to a smaller function,
since I want to work toward a better
approach for redrawing the topic list.

Also, since the new function is now
purely data-oriented, it will be a
bit easier to test various edge cases.
2020-01-22 14:31:33 -08:00
Steve Howell 5f41d99422 topic list: Simplify active_topic code.
There's no real need to pass this through the
stack like we do.
2020-01-22 14:31:33 -08:00
Steve Howell 1eab59d1b5 topic list: Kill off "no more topics".
If you clicked for no more topics and then the server didn't find any,
we once had code that would say "No more topics" in light gray at the
bottom of the topic list.

The feature appears to have been broken by some detail in the
`self.dom` refactoring.  More importantly, it's not clear it's useful
as opposed to clutter.

Since we added the `stream.first_message_id` feature, it's now very
rare for the `more topics` option to appear when there aren't in fact
older topics that could be fetched.  In cases where there are not, the
UI is still clear about what's happening -- it shows a loading
indicator and then displays a list of topics that doesn't have
anything new.

So we're removing this feature; we can re-add it without too much
difficulty if user feedback in the future suggests it would be useful
after all.
2020-01-22 14:28:59 -08:00
Steve Howell 613b8ade20 topic list: Add is_active_topic to template.
The only place we ever set active-sub-filter is
right after we build the template, so there is
no reason to have it be a separate step.

(I made a similar fix to pm_list recently, and
this helps set the stage for doing vdom-like
stuff.)
2020-01-22 14:28:50 -08:00
Tim Abbott e459a4b7a7 topic_list: Rewrite unzoomed rendering to be readable.
The previous logic was a bit byzantine, making a lot of inferences
based on which conditionals had already been processed that made it
hard to read.  This simple function approach promises to be more
readable.
2020-01-22 14:28:50 -08:00
Tim Abbott 37b563b82f topic list: Don't include muted topics in "more topics" count.
This is for consistency with how we show unreads in muted topics at
the stream level, avoiding distracting users with the appearance of
unread messages in muted topics that they've made clear they are not
interested in.

Arguably, we should show a faded count if there are unreads on muted
topics (but none on unmuted topics), but that seems somewhat complex
to maintain, and we'd benefit from user feedback to make an effective
decision on whether it'd be an improvement.

Fixes #13676.
2020-01-22 14:28:50 -08:00
Tim Abbott 6f4f6fde79 topic list: Skip muted topics when not zoomed.
I think this probably matches users' expected behavior that muted
streams shouldn't get in their way unless the user is actively looking
for them.  If a user has a lot of muted topics with active traffic
(e.g. because topics corresponding to channels in a mirrored Slack
instance), they would previously find their 5 slots cluttered with
those muted topics even if there were unmuted topics with unread
messages.

Fixes #13677.
2020-01-22 14:28:50 -08:00
Rodriq 8d466f6a25 docs: Update API docs usage example.
This docs on writing API docs usage example hadn't been updated to use
generate_code_example(curl) after we introduced that feature.
2020-01-22 12:00:10 -08:00
Mateusz Mandera 8dd95bd057 tests: Replace httpretty with responses.
responses is an module analogous to httpretty for mocking external
URLs, with a very similar interface (potentially cleaner in that it
makes use of context managers).

The most important (in the moment) problem with httpretty is that it
breaks the ability to use redis in parts of code where httpretty is
enabled.  From more research, the module in general has tendency to
have various troublesome bugs with breaking URLs that it shouldn't be
affecting, caused by it working at the socket interface layer.  While
those issues could be fixed, responses seems to be less buggy (based
on both third-party reports like ckan/ckan#4755 and our own experience
in removing workarounds for bugs in httpretty) and is more actively
maintained.
2020-01-22 11:56:15 -08:00
Tim Abbott 445a11753b docs: Update links to Django documentation.
Django 1.9 is no longer on the web.
2020-01-16 16:00:03 -08:00
Tim Abbott c113d74daf docs: Rewrite the guide on using the development environment.
This correct various inaccuracies and adds a bulleted list structure
for better clarity.

I think there's a lot more that could be done here in the form of
linking to other pages, discussing restarting `run-dev.py`, etc.
2020-01-16 15:57:08 -08:00
Chris Bobbe b3901c830b docs/development/using: Add link to ./authentication.
Added a link from docs/development/using ("Using the Development
Environment") to ./authentication ("Authentication in the development
environment") to help people working on the authentication systems
or anyone who needs an API key.
2020-01-16 15:41:12 -08:00
Chris Bobbe 6c3fcc252e docs/development/using: Small wording change.
An attempt to make the point about Django/Tornado server processes
auto-restarting less wordy.
2020-01-16 15:41:12 -08:00