New York is a “one-party consent” state, which means that you can record conversations if you are physically in the state and are taking part in the conversation, or if you have permission from one of the parties in the conversation.
If you are the person taping a conversation with someone else, it is obvious that you agree to the recording, hence it is legal.
Recording a call you are not participating in is generally illegal.
If you are just filming, then when you are lawfully present in any public space, you have the right to document, including to photograph or record, anything in plain view, including government buildings and the police. On private property, the owner may set rules about photography or video. Once you are off the street, you can be considered in a private place.